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ABSTRACT 

Many production facilities rely on the intuition of the 
quality asSurance inspector for determining what will be 
visually inspected and for what length of time on a finished 
PeOGuGCt. In this thesis a model for visual inspection of a 
finished product is developed. The key parameters in the 
model are the probability of worker error, the probability of 
inspector error, and the cost of system error. Paired 
comparisons of error phenomena from operational personnel are 
converted to probabilities using the indirect numerical 
estimation technique. 

The model is used in a goal program to optimize the use 
of inspector time in a production facility. The model and the 
goal program are applied to a military problem to demonstrate 
the broad applicability of the methodology. The military 
problem is the allocation of inspection time before firing an 
artillery weapon to insure accurate and timely delivery of 
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I. INTRODUCTION 


A. GENERAL 

Assume that all the resources needed for a particular 
production process are already purchased. They represent sunk 
Sesets to the production facility. The only cost of doing 
business then is the cost incurred by producing unacceptable 
goods. If finished goods are flawed or late when delivered 
to the consumer in a market economy then the producer 
eventually loses market share. Continued loss of market share 
leads to annihilation of the production business. Thus, the 
decision maker's task is to allocate the available resources 
to achieve timely delivery of error-free products. This is 
the situation in which some mid-level managers’ find 
themselves. In other words, given little or no control over 
the resources on hand, they are to perform as well as possible 
Pee are not to fail. 

The typical response to the mid-level manager's production 
problem is to employ part of the allotted labor resources as 
quality assurance inspectors. Use of labor resources as 
inspectors rather than producers decreases production. The 
time required to conduct the inspections slows the remaining 
production. The use of the inspectors, however, decreases the 


number of flaws in the finished product. The manager faces 


a tradeoff between two favorable objectives: better finished 
products versus more finished products. 

Assume that all worker errors in the production process 
that can lead to a flawed finished product can be identified. 
Assume that each worker error, (where i = 1,...,m) is 
independent and mutually exclusive of each other. Therefore, 
given that a flawed product is found bearing the effects of 


one error from the production process: 


(1-1) PY ( Crron, = = see 


h- 
it 
lets 


Assume a cost distribution can be identified for error. 
From those distributions assume that mean costs can be 
derived. Now the expected cost of doing business (EC) per 


each item produced can be written: 
m 
(1-2) S P (error,) **C (ennon,) —-sae 
SUE 


where m is the number of independent errors 
P is "“prebabi dat 
C is "mean cost" 

The probability for a flawed finished product due to error 
on task, [ P (error,) J is the probability that boc we 
worker erred on the task and the inspector failed to identify 
the worker's error. The inspector's ability to positively 


identify flawed products as being unacceptable increases over 


time. Naturally, the cost of doing business can be reduced 
if the allocation of available inspector time can be driven 
toward errors that occur more frequently and which have 
greater cost. How should the available inspection time be 
divided between error-prone tasks to best reduce the cost of 
doing business? The focus of this thesis is how to allocate 
the available inspection time to best accomplish the manager's 
multiple objectives of speed and thoroughness. 

Goal programming iS a convenient tool for simultaneously 
satisfying several objectives. The first half of this thesis 
proposes a methodology for allocating inspection time based 
on goal programming. The second half uses the artillery gun 
chief's inspection problem to illustrate the methodology. 

In offering a solution to the allocation of inspection 
time problem, the thesis uses several timely, efficient, and 
sound techniques to derive a deterministic probability for 
worker failure on each error-prone task, the expected time 
that an inspector spends inspecting each task, and the cost 
distribution of error on each task. The hope is that these 
techniques are sufficiently convenient that they can be 
readily adopted by mid-level managers in their allocation of 


inspection resource problems. 


B. DEFINITIONS 

Some terminology iS peculiar to the study of human 
reliability. Other terminology is peculiar to the U.S. Army 
and the field artillery. Appendix F contains a glossary of 


those terms. 


C. ORGANIZATION OF THE THESIS 

Chapter 2 presents the experimental results and theories 
that are the underpinnings of the proposed methodology for 
sOlving the allocation of inspection time problem. The 
chapter documents alternative techniques by which error rates 
and time estimates may be derived other than by empirical 
observation. The chapter reviews a technique for estimating 
costs of errors and discusses the attributes of goal 
programming. 

Chapters 3 through 5 present the problem that illustrates 
the proposed methodology. Chapter 3, Background tO “Gia 
Artillery Problem, describes the allocation of inspection time 
problem as it applies to the gun section and the artillery 
fire support system. The discussion of artillery explains the 
tasks of the various crewmen and the errors they make. 
Chapter 4 develops a model for the artillery gun chief's 
problem. Chapter 5 applies the methodology to an active army 
unit at Fort “Ord, Cali teen ae Chapter 6, Conclusions and 


Recommendations, proposes other applications Of this 


methodology and suggests areas for future research. The 
appendices give the output from the computer runs discussed 
in Chapter 5, the surveys of respondents, and step-by-step 
procedures for computing the solution to the artillery gun 


chief's inspection problem. 


Il. DEVELOPMENT OF A VISUAL INSPECTION MODEL 


A. GENERAL 
i Need for a Model over Intuition 
Often the allocation of inspection time to various 
inspection tasks is determined by the inspector. It we 
generally expected that an inspector will employ his full 
experience, knowledge, and intuition when deciding how to 
spend his time on the various visual inspection tasks he must 
perform. Intuition is inadequate for such tasks as allocating 
inspection time. Hogarth identifies the primary weakness in 
reliance on intuition in general: 
One could suppose that through intuitive judgment people 
are able to capture important information which escapes 
codification and is thus not amenable to statistical 
analysis. Indeed, intuitive judgment does capture much 
information, however, it is inconsistent. Moreover such 
inconsistency necessarily attenuates human predictive 
validity. In other words, although humans may on occasion 
exhibit high accuracy in judgment, inability to apply 
judgmental rules consistently across a series of cases 
means that on average models outpredict people. [Ref 2, 
p.194] 
Over the long run, a model that incorporates all available 
knowledge regarding inspection phenomenon and allocates 


inspection time accordingly will outperform any inspector's 
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A mathematical model of the inspection process offers 
many advantages over reliance on the intuition of an 
inspector. First, a model offers a means to understand the 
underlying causes and effects occurring in the inspection 
phenomena. Reliance on inspector's intuition offers no 
alternative method to solve inadequacies should intuition 
start failing. With a model there is the means to test 
alternative solutions. Gass states, 

It (a mathematical model) enables us to evaluate the 
effects of a change of one variable on all the other 
variables; it provides us with a quantitative basis to 
sharpen and evaluate our intuition of the process under 
investigation... Mathematical models enable us to bring 
some semblance of scientific methodology to areas of 
Gecision making heretofore characterized by intuition and 
experience. [Ref 42, p.26-27] 

Morse and Kimball write, "Operations research is a 
scientific method of providing executive departments with a 
Quantitative basis for decisions regarding operations under 
their control {Ref 19, p 7]." Building effective mathematical 
models of the inspection process allows managers to take 
control of their inspection resources and make optimal use of 
them in support of production facility goals and values. If 
inspectors are not brought in to support the goals and values 
of the production facility they are apt to pursue their own 
set of goals and values. Hillier and Lieberman allude to what 


then tends to happen ",..(there is) a tendency for many 


components of an organization to grow into relatively 


autonomous empires with their own goals and value systems [Ref 
oP, @ ee ee 

Inspector time is a resource of the manager and, as 
such, it should support the production facility's goals and 
values. If the manager can quantify his goals with respect 
to product quality then goal programming allows him to model 
his inspection time problem. The model gives him the means 
to simultaneously consider his goals and allocate inspector's 
time to satisfy those goals. 

2. Modeling of the Inspection Process 
The purpose of the inspection process is to identify 
worker errors that affect the system. The model of the 
inspection process must incorporate the effect that the errors 
potentially have on the _ system. Thus, a large part of 
modeling the inspection process iS error analysis. 
The importance of an error is directly related to its 
potential for unacceptable system effects. The Nuclear 
Regulatory Commission (NRC) makes this point in regard to 
errors in general: 
The importance of an error is a function of its frequency, 
the efficiency of the (inspector) for the error, and the 
likelihood and severity of the potential consequences of 
the error... [Ref 14, p 4-17]. 

In applying this same concept to the realm of human error, 


Pickrel and McDonald write: 


The extent and cost of efforts to eliminate sources of 
human errors should be commensurate with: 


1) the frequency with which the error is expected to 
OGeCuUr ; 


2) the frequency with which a (system) failure will 
occur as a result of the error; and 


3) the possible consequences’ of the failure 
condition. [Ref 27, p 649] 


Swain points the direction for the modeling of the inspection 
process when he writes: 


Errors are important only if they lead to important 
consequences. Thus, the steps in error analysis are 


a) Identify the consequences of interest; 


b) Identify all errors that can lead to those 
consequences; 


c) Determine error rates for each class of errors; 


qd) Identify the likely contributing factors. 
fret 10, p.i4] 


The probability of worker error, the probability of associated 
inspector error, and the cost of associated product flaws are 
the parameters needed to establish the relative importance of 
the different errors. These parameters must then be brought 
together in one model to represent the inspection process. 

This chapter reviews the rationale for using the 
Opinions of subject-matter experts to both estimate the 
probability for worker error and to estimate the time required 
to inspect for that worker error. Theories and experiments 


dealing with the probability for inspector failure as a 


dealing with the probability for inspector fajlure as a 
function of time to inspect and a method for cost estimation 
of system error are reviewed. 

Another area discussed in the model development is the 
importance of internal consistency in judgments made by 
subject-matter experts before their subjective evaluations 
have merit. Saaty's Analytical Hierarchy Process [Ref 17] is 
reviewed for its method of evaluating the internal consistency 
in a respondent. Also, in this chapter, goal programming is 
reviewed as a means to model problems such as the allocation 
of inspection resources problem. The goal program that 
achieves this allocation of inspector time is relatively 
Simple and straightforward. The challenge is developing the 
probabilities for error and the costs of error required by the 


model and the goal program. 


B. THE PROBABILITY OF CREW ERROR 
1. Rationale for Subjective Evaluations 
Research into human reliability and error analysis led 
to work by the Nuclear Regulatory Commission (NRC) in its 
evaluation of human error probability in the operation of 
nuclear power plants. The NRC is at the forefront in research 
on human error and design of functional, safe systems that 


incorporate human error rates. 


10 


— 





Traditional thinking is that personal opinion of 
people with working experience is worth little as a basis for 
good operations research work. Morse and Kimball were 
emphatic about this when they helped found operations 
research: 

The opinions of a few dozen persons who have _ had 
operational experience provide an extremely shaky 
foundation for any operations research...The need for 
unbiased, impersonal facts, not opinions, must always be 
borne in mind. [Ref 19, p 8-9] 
This influence is still clear today as there is often an 
aversion to using operational experience as a basis for 
operations research. Echoing the sentiments of Morse and 
Kimball is the following more recent comment by Von 
Winterfeldt and Edwards that typifies the views of many 
systems analysts about the appropriate way to assess 
probabilities: 
By far the most common practical procedure for assessing 
probabilities is to collect a large set of observations 
and then to use a relative frequency. This procedure is 
tedious but the existence of empirical science testifies 
to its success.[Ref 29, p. 110] 

Whereas probabilities for equipment failure may be 
best established by the strict empirical approach, it is 
unlikely that probabilities for human failure (human error) 
can be gathered in that manner. Swain and Miller provide a 


discussion of the reasons human error data generally cannot 


be obtained through empirical observations [Ref l, p 226]. 
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Horley and Giordano give a graphic example of failure of the 


empirical approach to obtain human error data on artillery gun 


sections in U.S. Army tests conducted at Fort Hood, Texas [Ref 


15, p.6/7276)s Swain and Guttman discuss why the empirical 


approach to generating human error data has not worked in 


nuclear power plants: 


The collection of relative frequencies (for human error 
rates by empirical observation) is impractical for several 
reasons. 


First, error probabilities for many taskKS are very 
small. Therefore, in any reasonable time period, not 
enough errors will occur to permit the fitting of a 
meaningful distribution to the observed error relative 
frequencies. 


Second, usually there is some penalty associated with 
the commission of an error, which discourages the 
identification of individuals who commit the errors. If 
such identification were insisted upon, there would be 
some unknown, but sizeable, number of errors not reported. 


Third, the administrative problems and costs of 
recording and analyzing errors by person would probably 
be unacceptable. [Ref 14, p. 6-2] 


There are numerous activities in need of hum 


reliability data for enhanced man-machine system design. 


an 


In 


most of these activities there is extensive operational 


experience, but there is little or no human error data. 


Expert judgment provides a means to generate human error data. 


Swain states, 


The use of collective judgment by subject-matter experts 
is one way to make up for the paucity of empirical error 
rate data. Studies indicate that it would be feasible to 
have a number of people with detailed knowledge of various 
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tasks in an industrial setting rank-order tasks in terms 
of error-likeliness. [Ref 10, p.101] 


Miller and Swain briefly describe how the methods that use 
expert judgment to generate human error data are structured: 
Subject-matter experts judge the relative or absolute 
likelihood of error for several task descriptions, and 
these responses are converted to usable human error 
probabilities by mathematical scaling techniques. [Ref l, 

pe 238 | 

The NRC endorses four methods for eliciting expert 

judgment input to estimate human error probabilities. The NRC 
makes the following exhortation in regards to these methods: 

Make the best use possible of current procedures and 

information, even though they may be flawed...These (the 

four methods endorsed by the NRC) are based on sound 

psychological theory and empirical support that indicate 

their potential usefulness. And as a practical matter, 

they can be used now to obtain needed human error 

probabilities. [Ref 14, p 8-6] 
The NRC approved methods are the paired comparisons procedure, 
ranking and rating procedure, direct numerical estimation 
procedure, and indirect numerical estimation procedure. Swain 
and Guttman [Ref 14, p.8-6 to 8-13] and Seaver and Stillwell 
[Ref 28, p A-2 to A-51] provide detailed explanations of each 
of the NRC approved methods. 

2. Experts to Supply Expert Judgment 
Embrey noted that his whole approach to developing 

human error probabilities by subjective input was contingent 


on judges. He writes: 


The success of the approach is likely to be critically 
dependent on the knowledge and experience that the judges 


is 


possess regarding the tasks they are asked to assess. For 
this reason it is very important that strenuous attempts 
are made to obtain appropriately qualified judges. [Ref 
Zoe Die SOY 

In commenting about eliciting judgments of uncertainty (or 


probabilities) from people, Von Winterfeldt and Edwards write: 


If a numerical meaSure of uncertainty iS an opinion, it 
characterizes a person as well as an event; we want to 
elicit from the right person. Who is that?..[Ref 29, p. 
Jt he 
The "right person" is the one who has the best understanding 
of the phenomenon under investigation and who can best relate 
that understanding in terms that can be quantified. 
Swain offers that the worker or inspector himself is 
an expert on error rates: 
Subject-matter experts are persons who are especially 
knowledgeable about the tasks for which error rates are 
to be derived. Thus, if one wanted to derive error rate 
estimates for new tasks in an assembly plant, the experts 
would be persons who perform or had performed Similar 
tasks in an assembly plant. [Ref 10, p.101] 
Saaty agrees. Saaty not only allows that the worker or 
inspector is an expert in the operational aspects of his work, 
but he implies that he also is the best person for relating 
to others phenomena about that work: 
We generally agree that people who experience a phenomenon 
firsthand are the ones who can best shed light on our 


understanding of it; indeed, knowledge derived from 
experience is basic to all understanding. [Ref 17, p. 9] 
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Lai, Shenoi, and Fan suggest that workers and inspectors are 
specifically well-equipped to estimate probabilities 
concerning errors and hazards: 
One of the major problems facing many quantitative risk 
assessment studies is the lack of proper data about the 


probabilities of occurrence of hazardous events. Often, 
the only alternative is to employ rough estimates provided 


by operations personnel.... Human beings often possess 
quality knowledge that arises out of their insights and 
experiences. When the safety of a system is being 


examined this expertise has the greatest weight. [Ref 40, 
p.136) 


While the worker and inspector are certainly experts 
on the operational aspect of the human error phenomenon that 
is a subject of this study, the question remains whether they 
can relate this expert understanding in terms that permit 
@w@encification. 

3. Indirect Numerical Estimation Technique 

Miller and Swain write that a major disadvantage of 
methods that use expert judgment to generate human error data 
is that the expert often lacks sufficient familiarity with 
probabilities to be able to give meaningful input [Ref l, p. 
244]. Persons with operational experience, such as workers 
and inspectors, are often not qualified to give a direct 
numerical estimate of a probability. Paulos states, "An 
appreciation for probability takes a long time to develop. 
meer /, p.i33)]." Morse and Kimball warn, "People seldom 


estimate random events correctly...their opinions are nearly 


ies 


always unconsciously biased [Ref 19, p. 8]." Swain cautions: 
It is not suggested that (subject-matter) experts be asked 
to make estimates of actual error rates for tasks. Such 
estimates would tend to vary widely and would generally 
underestimate the true error rates. Some subject-matter 
experts simply would refuse to make such estimates. [Ref 
Ore Dp LTO 

Hogarth states, 
People do have difficulty in expressing their degree of 
knowledge in the precise quantitative form demanded by 
probability theory. Numerous studies have shown 
fallibilities in intuitive attempts to guess probabilistic 
relations from given facts. [Ref 2, p.192} 

Embrey shows that, when giving uncertainty judgments, 
respondents do significantly better when dealing with 
likelihood ratios and odds rather than probabilities. Embrey 
concludes, "that judges (subject-matter experts) are best at 
making simple directional judgments of the form ‘task A is 
more likely to be successful than task B' [Ref 28, p 9J." 
Swain's work supports Embrey's. Swain adds: 

Persons who will not guess at the probability of a 
particular event will readily rank-order several events 
in terms of increasing difficulty, hazard, on weeheu 
dimension, and they will do this ranking with great 
confidence. [ReteilO, p.10m 
Miller and Swain comment, "Humans are very capable of making 
qualitative judgments along one dimension [Ref l1, p. 238]." 

Estimation methods for human error probability that 

allow a person without probability-estimating skills to relate 


his expert understanding of error phenomena by simple 


directional judgments could be very useful. Of the four 
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expert judgment methods proposed by the NRC, two methods are 
based upon simple directional judgments: paired comparisons 
and indirect numerical estimation. 

A problem with paired comparisons is the large number 
of judges required to obtain valid estimates of the task 
positions on a subjective scale. The relative frequency by 
which one task is rated more likely to fail than another by 
a large pool of experts is what leads to a scaled value for 
that task. If there are only five judges providing input then 
there are insufficient observations to be significant. 

The indirect numerical estimation method of obtaining 
human error probabilities from expert judgement is a robust 
procedure and is the primary procedure for estimating human 
error rates pursued in this thesis. Like paired comparisons, 
it depends mostly on human qualitative versus quantitative 
judgments. The indirect numerical estimation method asks the 
respondent two questions concerning task failure in his area 
of expertise: 


1) Is this task error more or less likely than that task 
error 7 


2) How much more or how much less likely is the first 
task error versus the second task error? 


A detailed discussion of the conversion of these expert 


judgments to human error data by the indirect numerical 
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estimation method is given by Seaver and Stillwell [Ref 12]. 
A summary of that process is given in the example below. 
Using the oone-by-one comparisons given by a 

respondent, a subjective scale of likelihood is generated. 
Assume a respondent gives these likelihoods for four mutually 
exclusive, independent task errors that encompass the entire 
set of errors that can occur on a given system: 

Task A error is twice as likely as Task B error. 

Task B error is just as likely as Task C error. 

Task C error is three times as likely as Task D error. 
The likelihood of task error for all tasks can simply be 
written in terms of any one task error. In the example, use 


Das the base scale: 


I 


Task A error 6 X (Task D error) 


Task B error 3 X (Task D error) 


Task C error 3 X (Task D error) 
Task D error = 1 X (Task D error) 
If the probability of any one task error is Known or estimated 
from some other source (Task D for example) then the 
probabilities of all four events are known. 
If the error probabilities are normalized, then the 
probability for error in any one task is Known, given that an 


Crreonmmoccurs: Normalization takes place in the example 


problem: 
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PouetecoDereo + 1D = 1.00 
D = .0769 

Therefore, given an error occurs on the system, the 
probability that it was: 

Task A error is 46% 

Task B error is 23% 

Task C error is 23% 

Task D error is 8% 
These probabilities for task error apply to one respondent. 

The next topic concerns the number of respondents 

needed to generate good point estimates of the error data 
being studied. Von Winterfeldt and Edwards state, "Both 
theory and evidence teach us to use more than one respondent 
whenever we can. [Ref 29, p.113)." Guttman and Swain, 
commenting on the indirect numerical estimation technique, 
cite the work of Von Holstein in stating, "as with direct 
numerical estimation, little is gained by using more than six 
experts [Ref 14, p A-46]." Hogarth writes: 

..-judgments between individuals are likely to be 

correlated. Thus one is often averaging redundant 

information with the result that beyond a certain point 

gains in adding experts to a group are small. It has been 

estimated that in most predictive situations involving the 

use of experts there is little point in averaging more 

than about ten persons and that six or seven will often 

be sufficient. [Ref 2, p 195] 


The next subject is the aggregating of several 


individual likelihood estimates to a single estimate 
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applicable to the entire production facility. Many schemes 
exist for combining several likelihood estimates into a single 
estimate. Von Winterfeldt and Edwards offers this insight: 
Since proper scoring rules are convex functions on the 
probability simplex, the score of the average of 
individual probabilities will necessarily be better than 
the average of the individual's scores. [Ref 29, p.133] 
The authors continue with this straightforward recommendation: 
Simply elicit the desired uncertainty measures from each 
member individually.... Transform the results into 
probabilities and average them. The odds seem excellent 
that, if you do anything more complex, you will simply be 
waSting your effort. [Ref 29, p.136] 
This thesis pursues the method recommended by Von Winterfeldt 
and Edwards. 
4. Consistency in Expert Judgment 
Workers and inspectors are typically trained to simply 
perform the tasks required in their jobs. They are not 
routinely trained to make quantitative judgments about 
phenomena relating to their work. When asked to make 
quantitative judgments it can be expected that some personnel 
are overwhelmed with the requirement and thus fail to provide 
valid eine. It would be helpful to have a means to screen 
the incoming data to eliminate Surveys of those respondents 
who were incapable or unwilling to provide valid data. 
Consistency on the part of a respondent iS some 


measure of objectivity, internal logic, and operational 


experience. Saaty defines the term: 
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The meaning of consistency is that the intensities of 
relations among ideas or objects based on a particular 
criterion justify each other in some logical way. (Ref 
17, p 18) 
Some measure of consistency is necessary to establish the 
credibility of the expert whose judgments will eventually 
affect a final decision. Saaty states, "We do not want the 
decision to be based on judgments that have such low 
consistency that they appear random (Ref 17, p.82)." 
Stillwell and Seaver add, "High levels of inconsistency would 
suggest the entire judgmental process is suspect (Ref 13, 
peo 2). " A way to identify inconsistent respondents is to 
require them to make more pairwise ratio judgments’) than 
absolutely necessary and then use those judgments to check the 
respondent's internal consistency. Seaver and Stillwell 
describe how this works for the indirect numerical estimation 
method of obtaining human error probabilities: 
To obtain a complete set of probabilities for N events 
only requires a minimum of N-1l such judgments and an 
independent estimate of the human error probability of 
one event. Additional judgments, however, may be obtained 
as consistency checks. [Ref 12, p A-46] 
When more pairwise ratio judgments are made by a respondent 
than is absolutely necessary there will most likely be some 
inconsistency. Stillwell and Seaver remind that, "Some 


inconsistencies enter into all judgmental processes. (Ref 13, 


Deo)" Saaty argues that some amount of inconsistency is 


Za 


necessary aS humans adapt to an ever-changing world. Saaty 
states: 
We may not be perfectly consistent, but that is the way 
we tend to work. It is also the way we grow. When we 
integrate new experiences into our consciousness, previous 
relationships may change and some consistency is lost. 
[Ref «7, «pe 2s) 

A level of consistency is desirable to establish 
respondent validity and expertise. A level of inconsistency, 
however, is to be expected among human judgment. Saaty 
comments, "As long as there is enough consistency to maintain 
coherence among the objects of our experience, the consistency 
need not be perfect [Ref 17, p.82]." There is a definite 
need to establish an acceptable level of inconsistency in a 
judge, beyond which the judgments are discarded as invalid. 

To obtain some measure of consistency Saaty suggests 
the development of an (N X N) reciprocal matrix where N 
represents the number of items for comparison. A respondent 
makes a ratio comparison between every two possible items such 
that a total of (N) X (N-1)/2 judgments are made. The (N X 
N) matrix is filled in with ones down the main diagonal, the 
(N) X (N-1)/2 judgments fill the lower triangular matrix below 
the diagonal of ones, and the reciprocals of the judgments 
fill the upper triangular matrix above the diagonal of ones. 


The maximum eigenvalue (A,,,) for this matrix is a 


measure of consistency of the judgments in the matrix. Saaty 
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proves that, "A positive reciprocal matrix is consistent if 
and only if A,,, is equal to N. (Ref 16, p.181)" Saaty then 
turns A,,, into a consistency index (CI) by the following 
operation: 

(2-1) arene), [| N-1 ) = "Cr. 
Saaty suggests that comparing the CI of a set of respondents’ 
judgments against that CI found by a randomly generated 
reciprocal matrix should give a general measure of the 
consistency called the consistency ratio (CR): 

ee Cleese, ) /. (Clrvaram Btzi,) © CR. 
The random entries in the random reciprocal matrix are chosen 
by computer from the range of possible values used by the 
respondents on their original Survey. Several hundred random 
matrices are computed with respective CIs for each. Those CIs 
are then averaged to obtain a mean CI for a random matrix. 
Appendix D shows the table for judgments limited to a one to 
nine scale and describes the generation of the table in more 
detail. 

Saaty provides a general rule for acceptable 
consistency ratio values with which he had success in his 
work. Saaty maintains that a consistency ratio less that 0.1 
demonstrates acceptable consistency and that a respondent 
providing such consistency is giving judgments’ worth 


considering in decision-making. [Ref 17, p. 83] 
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A: disadvantage of Saaty's method is the number of 
judgments required of each respondent. Ina 4X 4 matrix an 
expert makes only six judgments [(4 X 3) / 2]. In a10O X 10 
matrix an expert must make 45 judgments [(10 X 9) / 2]. Saaty 
Suggests extensive use of hierarchies so that the number of 
judgments are reduced. For example, if the 10 X 10 matrix 
could be partitioned to sets of four, three, and three for 
pairwise ratio comparisons then the number of judgments is 
reduced from 45 to 15. Figure 1 depicts the reduction of 
required judgments that is accomplished by use of a hierarchy. 

Saaty's method is the method pursued in this thesis 
1senS establishing sufficiently consistent input fon 


consideration in decision making. 


cz THE PROBABILITY OF INSPECTOR ERROR 
1. Inspectors Make Mistakes 

Traditionally inspectors are viewed as a panacea for 
production problems. Many production and quality assurance 
plans are premised on an assumption of 100% accuracy on the 
part of inspectors. Inspectors are not perfect. Swain makes 
this clear, 

There iS no point in arguing whether people should make 


errors. We all do. Human error rates, like material 
failure rates, are merely facts [Ref 10, p.5]. 
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Figure 1 - Reduction of Pairwise Judgments 
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Drury makes this call to action: 
Intrinsically, man is rarely perfect in either detection 
or diagnosis. . .This has led in recent years to studies 
which have demonstrated the effects inspector's errors can 
have. . .and the need to take account of realistic values 
of inspector's errors. [Ref 9, p.11] 
To decrease the error rate some form of data collection is 
necessary to make a quantitative assessment of inspector 
error. Bennett emphasizes this point: 
So what must be done to achieve quality objectives given 
that inspection error is to be ever present? The answer 
is simple. The quality control engineer must be able to 
accurately measure these errors and then be able to design 
for them. [Ref 9, p.1] 


2. The Direct Relationship: the More Time Inspector 
Takes, the Greater the Inspector Accuracy 


An appealing method for viewing inspector error 
probability was introduced by Wreathall in work for the NRC. 
Wreathall's Operator-Action Tree method (OAT) is depicted in 
Figure 2. 

The OAT approach is based on the assumption that human 
response to an environmental event consists of three 
activities: 

1) @eperceptien- 

2) diagnosis, and 

3) response. 


The basic operator-action tree is based on the potential 
for error in each of the three activities. 


The second major assumption is that time available for 
diagnosis is the dominant factor determining the 
probability for failure. That is, given a short period 
of time, people will fail to diagnose a situation 
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correctly more often than when given a longer period. [Ref l, 
Ee 2oo | 


BASIC OPERATOR ACTION TREE 


OPERATOR OPERATOR OPERATOR 
EVENT OBSERVES DIAGNOSES CARRIES OUT SUCCESS 
OCCURS INDICATIONS PROBLEM REQUIRED RESPONSE FAILURE 


$ 


SOURCE: NUREG/CA-3010 


Figure 2 - Operator-Action Tree 


The intuitive appeal for the first assumption is overwhelming. 


] Th] 


An inspector must "see", or visually fixate, on the event. 
He must be trained sufficiently to be able to discriminate 
error in the object of his fixation. Finally, the inspector 
must be able to respond properly to overcome the error. 
Failure to accomplish any one of these three steps will result 
in inspector failure on the task. 

Not only does the second assumption, that less time 
available means more inspector errors, make intuitive sense, 
but there is strong experimental basis for it. Drury's work 
with flat glass inspectors [Ref 26, p.265], Buck's experiments 
with Landolt rings [Ref 9, p.169], and Rizzi's tests with 


flawed dot patterns on cards [Ref 25, p.279] convincingly 


demonstrate that inspector errors increase with less exposure 
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time to the object being inspected. Drury summed up his work 
and that of Smith and Barany [Ref 22] when he wrote: 
In all of the studies, a similar effect is observed, in 
that as more time is allowed to inspect each item, the 
probability of rejecting a faulty item increases whilst 
the probability of accepting a good item decreases. [Ref 
SZ 5 p.. 6] 
Drury suggests something akin to OAT: 
The inspection mechanism postulated is a visual search 
until either a potential defect is found or time runs out 
for that item, followed by a decision about the potential 
defect as to whether it is acceptable or rejectable. [Ref 
S259 ps Ol 
Just as the OAT method suggests, a visual search must take 
place until the inspector "sees" the prospective error. This 
Success of the visual search is no doubt a function of many 
factors. The experimental evidence reviewed points strongly 
to time available to the inspector as the most significant 
factor to whether the search will be successful or not. If 
the search leads to a prospective error then the inspector 
must decide whether to accept or reject the object bearing the 
prospective error. This decision involves experience of the 
inspector, remaining time available to the inspector, and 
costs and payoffs relating to finding errors. 
Smith and Barany found signal detection theory helpful 
in describing the decision an inspector must make [Ref 22, 


Peso). Drury also used signal detection theory to model 


inspector decision performance [Ref 32, p.2-6]. Chapman and 
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Sinclair give this opinion of the application of signal 

detection theory to their study of inspection work: 
Perhaps the most important thing to emerge from this study 
is the usefulness of the theory of Signal detection in 
industrial inspection situations both from a theoretical 
and a practical point of view. The theoretical value lies 
in the insights it provides into an inspection task, and 
the practical value arises from the fact that it allows 
economic justification for the application of ergonomics 
to inspection, and the rel-ative ease with which 
recommendations for improvement can be derived. [Ref 9, 
p. 241] 

The basics of signal detection theory were established 
by the pioneering work of Tanner, Swets, Birdsall, and Green 
between 1954 and 1966 [Ref 8, p.18]. Figure 3 is the common 
depiction of the theory applied to the inspector's decision 
[Ref 9, p.14]. The detection of signals is assumed by the 
theory to involve two processes: discrimination (d') and 
Gecision (8). Discrimination (d') requires the inspector to 
be capable of distinguishing between success and error on a 
task he is inspecting. 

Discrimination is affected by such things as the 
experience of the inspector (more experience tends to make d' 
larger) and the time available to perform the inspection (more 
time tends to make d' larger). Decision (8) requires the 
inspector to accept or reject what he sees. A decision (where 


the point $8 is between the two modal values of the Gaussian 


curves) is influenced by such things as the inspector's 
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Crtterton (8) 
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Figure 3 - Signal Detection Theory [Ref 9, p.14] 
subjective probability that an error will occur (if he thinks 
errors are very likely then he is more likely to reject) and 
his attitude toward risk. [Ref 8, p.18-21] 

Smith and Barany explain in terms of signal detection 
theory what transpires to cause a reduction in inspector 
accuracy with a decrease in time available. Smith and Barany 
write: 

If the pace of the inspection task is increased, then the 
inspector will have less time in which to make his 
observation, and his observing mode will tend toward that 
described as "blurred observing". [{Ref 22, p.300] 
The effect Or "blurred observing" is to decrease 
Giscriminability (d'). With discriminability (d') reduced 


there is more” overlap of the two Gaussian curves (see Figure 


3) and whatever decision (8) is made must result in either 
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more Type I or Type II errors. Smith and Barany conducted 
experiments which demonstrated what signal detection theory 
postulated. 
The experimental results from Drury [Ref 26]; Rizzi, 

Buck, and Anderson [Ref 25]; and Smith and Barany [Ref 22] 
noted above are for relatively simple inspection tasks along 
one dimension (flaws in sheet glass, numbers of holes in 
Gisks, dots on cards) as opposed to more complex tasks with 
a multitude of possible errors on several dimensions (circuit 
boards, silicon chips). For Simple inspection tasks the 
experimental results clearly show a direct relationship 
between inspector time to observe and inspector accuracy. 
Schoonard emphasized this point with an experiment where seven 
different classes of errors were possible on circuit boards. 
The inspection tasks for the seven classes were ranked from 
Simplest to most difficult. A deliberate slowing of the 
inspection time resulted in marked improvement on the more 
Simple inspection tasks but not so great an improvement on the 
other, more difficult inspection tasks. [Ref 24, p. 368-376] 
Drury predicted this finding by signal detection theory 
principles. In reference to simple inspection tasks Drury 
wrote, 

For simple discriminations (such as the Smith and Barany 

experiments) the discriminability will be high enough to 


give almost perfect inspection as the time allowed is 
increased. [Ref 32, p.7] 


Sil 


It is clear from experimental results that simple inspection 
task accuracy improves with time. 

The nature of the relationship between time taken by 
inspector and his accuracy was explored by both Rizzi, Buck, 
and Anderson [Ref 25, p. 279] and Drury [Ref 26, p.265]. 
Drury found an exponential relationship between the 
cumulative probability of locating a flaw in sheet glass and 
time taken to inspect. Rizzi, Buck, and Anderson conducted 
tests on subjects inspecting small cards with dots. Similar 
to Drury's results, Rizzi, Buck, and Anderson also found the 
inspector's exposure time to the object exerted an exponential 
effect on the probability of a correct detection. Figure 4 
shows the results of Drury's and Smith and Barany's work with 


the effect of time on the probability of inspector accuracy. 
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Figure 4 - Inspector Accuracy as a Function of Time 
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3. Limits to the Direct Relationship 
It would seem that there must be some limit to the 
direct relationship that more time spent by an inspector 
yields more inspector accuracy. This section reviews 
experimental results that confirm both lower and upper bounds 
to this direct relationship. 


a. Minimum Time is Limited by Time Required for 
Visual Fixation 


The lower bound for this direct relationship is 
the physical limitation of the human eye to focus and fixate 
on any visual stimuli and the brain to comprehend what the eye 
is seeing. Experimental results revealing what might be a 
minimum time for the eye to fixate and the brain to begin to 
function for inspection purposes are offered by Drury. Drury 
concluded from his experimental results that it took seven 
tenths of a second (700 milliseconds) for his subjects to open 
and focus their eyes and begin searching for errors. [Ref 26, 
p. 260]. During that initial time from zero to seven tenths 
of a second there is no increase in inspector accuracy. 

b. Maximum Time is Limited by Human Vigilance 

The upper bound for the direct relationship of 
more time resulting in more accuracy is the result of 
decreased vigilance that occurs when an inspector exceeds his 


attention span. Chapman and Sinclair [Ref 9], Kochhar and 
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Jaisingh [Ref 23], and Schoonard, Gould, and Miller [Ref 24] 
demonstrated upper bounds to the direct relationship that was 
found by Drury [Ref 26]; Rizzi, Buck, and Anderson [Ref 25]; 
and Smith and Barany [Ref 22]. Experiments by Chapman and 
Sinclair of experienced personnel inspecting poultry carcasses 
at a meat processing plant [Ref 9, p.231] and the work of 
Kochhar and Jaisingh with experienced inspectors checking 
products on a CRT screen [Ref 23, p. 44] are two examples of 
note. Both showed initial improvement in inspector accuracy 
as time was increased, just as Drury and others had noted. 
At a point, however, Chapman and Sinclair [Ref 9] and Kochhar 
and Jaisingh [Ref 23] reported a degradation in inspector 
accuracy when inspectors were made to take more time in their 
visual inspection tasks. Schoonard, Gould, and Miller 
reported similar findings with their experienced inspectors 
who were checking circuit boards for any of seven different 
Classes of errors that might be present. As Schoonard, Gould, 
and Miller increased the inspection time to 1.5 times the 
standard time the inspectors spent on their tasks, significant 
improvement resulted across all seven classes of error-types. 
With any additional increases over the 1.5 times the standard 
time for the inspectors, there was no improvement in inspector 
accuracy... Refs 245 pasoole Chapman and Sinclair offer a 


possible explanation for this degradation phenomenon with 
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increase in inspector time: 


...Guring the course of such a task periods of inattention 
occur which become more frequent or prolonged as 
presentation time increases.... A possible answer (for the 
prolonged periods of inattention) may be that the human 
organism maintains a homeostatic level of arousal, and if 
this is dependent on incoming stimuli, it could be argued 
that at the lower speeds additional stimuli are required 
which are obtained during the extended periods of 
inattention to the task. [Ref 9, p 249] 


Kochhar and Jaisingh offer this explanation of the decline in 
inspector accuracy resulting from increased inspection time 
available that was noted in their experiments: 
Error detection in vigilance situations of this type is 
dependent upon operator arousal levels. Thus the low 
levels of fault information and product pacing did not 
arouse the subject to the point where performance could 
be optimal.... It could be inferred that operator 
performance in inspection is related to arousal and in 
evaluating trade-offs between the different variables due 
consideration should be given to maintaining operator 
arousal. {Ref 23, p. 44] 
The phenomenon that more time beyond a certain critical point 
does not yield more inspector accuracy is real. Analysts 
cannot blindly assume that more inspector time will continue 
meme ld an improving probability for accuracy. To continue 
to allot more inspector time is counterproductive. 109) 
addition to an actual decrease in inspector accuracy, more 
inspector time costs the facility either the salary of more 


inspectors or reduced productivity due to a slowed production 


ime . 


So 


4. Significance of Schoonard's Findings 

Many visual inspection experiments’ have been 
conducted. Few of those experimental findings have direct 
application to the actual inspection operations on production 
lines. Schoonard, Gould, and Miller explain the shortcomings 
of the bulk of visual inspection experiments: 

Fundamental laboratory studies... have limited application 
to most actual inspection situations because they use 
discrete targets on homogenous backgrounds, whereas most 
inspection situations consist of poorly defined targets 
On non-homogenous backgrounds. [Ref 24, p 365] 

Another shortcoming of much experimental work is the 
hiring of students to perform the simple visual inspection 
tasks in the laboratory studies. The nature of most actual 
inspection tasks is sufficiently technical to require 
specially trained inspectors. The inspectors' frames of mind, 
motivation, and dedication have much to do with how they will 
perform when inspecting. The results will very 1ikely differ 
Significantly between experiments where fully paid and trained 
inspectors perform technical inspection tasks that they 
perform everyday and other experiments where students are paid 
some nominal fee to perform very Simplistic visual inspection 
tasks for a day. To extrapolate information from experimental 
work for use in analysis of actual inspection work, the 


experiment should closely resemble the actual inspection 


Situation in several key respects. 
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aa Estimating Time for Inspector to Inspect 
An estimate on the standard time that it takes an 
inspector to perform a viSual inspection task is necessary to 
establish the range of time over which an inspector can 
productively be employed on a task. In regards to eliciting 
estimates of times required to inspect for worker error the 
NRC writes: 


Estimation of performance times poses a more difficult 


problem (than human error probabilities). The best 
approach is to take measurements of actual or simulated 
task times. A second-best approach is to combine 


estimates of those who perform the tasks. [Ref 14, p. 6- 
1] 


Nothing precludes empirical measurement to derive the required 
time estimates of visual inspection tasks except the cost and 
difficulty of doing so. There may be several potential worker 
errors within the visual scope of the inspector at any one 
time. Measuring the time an inspector takes on any one visual 
task is complicated by the difficulty of the experimenter 
trying to track where the inspector's eyes are fixated. The 
costs of empirical meaSurement of times for viSual inspection 
tasks is large due to the fact that each facility must likely 
derive its own time estimates due to individual facility 
peculiarities. Those peculiarities involve, for instance, 
level of training of the inspectors, special equipment that 
inspectors use, and standard operating procedures within the 


macality. 


3% 


An alternative to empirical observation to derive time 
estimates on inspectors is a modification of the indirect 
numerical estimation technique. The standard inspector time 
on one task may be Known through empirical observation. All 
other tasks are compared to one another on a respondent survey 
£Gmam- The respondents make directional judgments between 
pairs of tasks that appear on the survey form. The judgments 
call ster: 


1) Which of the two visual inspection tasks requires more 
time to accomplish? 


2) About how much longer (in terms of seconds) does the 
one task take to inspect compared to the other task? 


Through simple algebraic manipulation estimates are derived 
from the survey as to the amount of time taken to perform all 
visual inspection tasks. This algebraic manipulation is given 
with sample data in Appendix C. These point estimates are 
averaged with the input of others from the same facility to 
yield a facility-wide estimate of time for an inspector to 


visually inspect every task. 


D. THE COSTS OF ERROR 
Many costs vie for the attention of the analyst and the 
executive. Swain writes; 
Most consideration must be given to those potential errors 
with an intolerable combined possibility of occurring, 


going undetected, and causing an unacceptable system 
consequence. [Ref 10, p.90] 
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An error that causes unacceptable system consequence iS an 
error which is potentially too costly in terms of the 
consequences of interest for the system. Swain writes of the 
difference between errors that cause acceptable system 
consequence and those that cause unacceptable system 
consequence: 
For some errors, the system consequences may not be severe 
and perhaps a somewhat high possibility of an uncaught 
error can be tolerated. Other potential errors might have 
such severe consequences that uncaught error probabilities 
of even one in a million could not be tolerated. For 
example, a pharmaceutical company could probably not 
tolerate the existence of one cyanide capsule in one 
million multivitamin capsules delivered to the public. 
[Ref 10, p.89] 
The analyst must develop ways to assess potential costs of 
errors so that the manager can allocate error catching 
resources with regard to the potentially most damaging errors. 
The reallocation of error-catching resources in an 
organization is a change in system design. It is important 
to remember that in a systems approach: 
-.-SuccesSful design changes may not always reduce the 
probability of an error-but they will reduce _ the 
probability of undesirable system effects due to such 
errors. [Ref 10, p 90] 
In the multivitamin example mentioned by Swain, a very 
successful design change may be one that increases’ the 
incidence of cracked capsules (a frequent error of small 


costs) in bottles, but absolutely assures that a cyanide 


Capsule (a rare error of monumental costs) never shows up in 
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a bottle of multivitamin capsules. It is essential that the 
potential costs of errors be estimated before resources can 
be allocated to reduce the probability of undesirable system 
effects. 

For any one type of error there is a corresponding 
distribution of costs. The U.S. Army suggests a quick method 
for establishing a cost distribution based on the Project 
Evaluating and Review Technique (PERT): 


One might well consider the possible use of subjective 
cost estimates based on PERT... 


Once it has been decided to employ the PERT technique 
to determine likely costs, it becomes necessary to 
estimate the most optimistic cost, the most pessimistic 
cost, and the most likely cost... 


These concepts fit in rather well with the idea of a 
generalized Beta distribution. An adequate statistical 
model of the Beta density to represent probable cost is: 


(2-3) f(c) = k(c - A)P * (B-c)4, A<<<B 
where: 
f(c) = probability density function of costs 
c = cost 
k = ae to make area under distribution curve 
(228) k= 1 / [(B-A)R Slee cate 
Bp = complete Beta function 
A = minimum cost 
B = maximum cost 
P,q = parameters determining the shape of the beta 
distribution [Ref 33, p.36-6] 
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From the PERT cost distribution a mean and standard deviation 


can be easily computed. 


(2-5 ) m = estimate of mean cost = (a + 4*m, + b) / 6 
(2-6) v = estimate of variance = (b - a)* / 36 
where: 
a = estimate of minimum cost 
b = estimate of maximum cost 
m = estimate of most likely cost 


Oo 


[Ref 33, p.36-7] 
See Appendix B for more details of this procedure. 

The PERT cost estimation technique requires expert opinion 
to generate estimates for the cost distribution. Moder and 
Phillips point to an immediate supervisor as one capable of 
providing estimates for PERT: 

The basis of PERT computations...depends on the judgment 
of the person in charge of the activity in question. He 
is asked to call on his general experience, and his 
knowledge of the requirements of the activity in question, 
to consider the personnel and facilities available to hin, 
and then to estimate the three (costs): optimistic, 
pessimistic, and most likely (costs). [Ref 34, p 204] 
E. GOAL PROGRAMMING 
1. Versatility and Utility 
Goal programming is a powerful mathematical 
programming technique which allows for models that very 


closely approximate problems in industry, business, and 


government. The power of goal programming iS wrapped up in 
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its ability to overcome human analytical weaknesses. Paulos 
writes about one of those weaknesses: 


There's a strong human tendency to want everything, and 
to deny that trade-offs are usually necessary...Trade-offs 
between...speed and thoroughness...are frequently muddled 
and covered with a misty gauze, and this decline in 
Clarity is usually an added cost for everyone. [Ref 7, 
p.t3)) 


The decision maker can often visualize and announce what he 
wants to accomplish in several competing areas. The level at 
which the decision maker wants to achieve is a goal. The 
several goals that a decision maker has are often very 
different. Budnick, Mojena, and Vollman write of the power 
of goal programming: 
Goal programming...allows for consideration of multiple 
goals. The goals may or may not be of the same dimension 
or unit of measurement. In addition, goal programming 
allows for consideration of conflicting goals. [Ref 6, p 
She) | 
When it comes to giving something up for something 
else the decision maker often balks. Bell, Keeney, and Raiffa 
note the quandary in which the decision maker finds himself: 
This is what the multi-objective problem is all about: 
the making of vexing value trade-offs. There is no magic 
formula for making these value trade-offs. The decision 
maker would, of course, like to do the best he can in each 
attribute, but it is not possible to maximize several 
things at once. If we reduce unemployment, it may be at 
the expense of increasing inflation... These problems are 


pervasive and are at the heart of many public policy 
controversies. [Ref 35, p.4] 
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Any method that helps coax the decision maker into making 
tradeoffs is a help. Goal programming is a method for 
attacking the multi-objective problem. 

A decision variable is established which is the entity 
over which the decision maker has some control or influence. 
The decision variable is often a resource. The amount of the 
resource that can be employed is limited by constraints on the 
system. Through some Known or hypothesized relationships 
varying amounts of the decision variable effect the levels of 
goal achievement. The relationships by which those decision 
variables affect the decision maker's goals are captured by 
mathematical models in the goal constraints. An achievement 
function mathematically combines the deviations from all 
goals. By minimizing or maximizing the achievement function 
through linear or nonlinear programming methods a level of 
resource (decision variable) use is determined such that the 
goals are most closely achieved. 

Goal programming calls for the satisfying of goals 
rather than the maximization of some single objective, as is 
often the case in standard linear programming. The decision 
maker states his goals, ranks them as to their importance, and 
weights them if necessary. The analyst determines the 
relationships between a controllable variable and _ the 


achievement of the goals. Goal programming gives the amount 
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of the controllable variable to expend to accomplish as 
closely as possible what the decision maker desires. 
Budnick, Mojena, and Vollman explain the satisfying 

of goals in this way: 

With multiple goals, all goals usually cannot be realized 

exactly. Goal programming attempts to minimize the 

deviations from these goals with consideration given to 

the hierarchy of stated priorities. [Ref 6, p 352] 
Ignizio refers to the greater applicability of multi-objective 
models to actual problems as a reason to pursue goal 
programming. Hillier and Lieberman present a concise review 
of goal programming [Ref 5, p 242-252]. 

z. Applied to Allocation of Inspection Time 
The overall allocation of inspection resources problem 

is one that is determined by executives in an organization. 
Take, for instance, an organization making an investment 
decision. A million dollar budget for quality assurance is 
being allocated. The decision of going with twenty visual 
inspectors or two visual inspectors and robotics is a decision 
far removed from the floor manager. it is not commonly within 
the manager's means to hire or fire inspectors or buy new 
inspection-aiding equipment. The manager must take what the 
organization provides him, whether it is two or twenty visual 
inspectors, and make the facility work to achieve quality and 


quantity standards. Basically, the only thing that the floor 
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manager may directly control in regard to inspection of the 
final product is the allocation of time that the inspector 
spends per visual task. 
a. Decision Variables 

The decision variables in goal programming are the 
variables over which the decision maker has some control and 
which have some influence over the achievement of the goals. 
In goal programming to achieve allocation of visual inspection 
resources the inspector time spent on checking the work of 
others is the decision variable ( X,, i= the number of visual 
inspection tasks). The purpose of goal programming is to 
determine an acceptable use of the decision variable. 

b. System Constraints 

A system is always limited in the amount of the 
decision variable that can be employed. This limitation is 
due to a limited supply of the resource or limited capacity 
of the system to employ the resource. In goal programming to 
allocate visual inspection time resources at least two system 
constraints exist: the minimum and the maximum amounts of 
time that can be spent productively on each of the inspection 
tasks. These limits are based on the experimental findings 


of Drury [Ref 26] and Schoonard, Gould, and Miller [Ref 24]. 
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c. Goal Constraints 
Goals reflect the objectives of the decision 
maker. With visual inspection there is usually a thoroughness 
goal and a speed goal that the manager specifies. These goals 
are elastic, allowing some deviation if insufficient resources 
are available to satisfy both goals. 


(2=7) => X, - TPOS + TNEG = TIMEGOAL 


: ak 
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re 


where m = the number of inspection tasks to be done. 
The right hand side of this constraint represents a goal in 
terms of time in which the manager wants inspections to be 
completed. The deviation from the goal which matters to the 


manager is captured by TPOS. 


m 
(2-8) = [ £(x,) * CCerror,) ] - ACCPOS + ACCNEG = ACCGOAL 
1= 


probability of inspector error as a 
function of time he spends on the 
inspection task, 


where f(x, ) 


C(error,) = the cost of error, in meters. 


The right hand side of this constraint represents a goal in 
terms of accuracy that the manager wants to be met. The 
deviation from the goal which matters to the manager is 


Captured by ACCPOS. Deviations in one goal versus another are 
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encouraged through weights that are applied to the respective 
deviations in the achievement function. 

The desires of the decision maker are often 
preemptive: one tier of goals must be satisfied before 
another tier can be considered. Hillier and Lieberman 
describe preemptive goal programming and the handling of 
goals: 

Consider the case for preemptive goal programming. Such 
a case arises when one or more of the goals clearly is far 
more important than the others. Thus the initial focus 
should be on achieving as closely as possible these first 
priority goals. After we find an optimal solution with 
respect to the first priority goals, we can break any ties 
for the optimal solution by considering second priority 
goals. When we deal with goals on the same priority 
level, our approach is just like the one described for 
nonpreemptive goal programming. [Ref 5, p.245] 

Assume a number of independent visual inspection 
tasks which must be accomplished properly for an error-free 
product to come out of the facility. Assume that a cost, a 
quantitative measure relating to loss of customer 
Satisfaction, can be associated with each error on a product 
that leaves the facility. If the manager can state a maximum 
tolerance cost, within which the confidence of the consumer 
is maintained, then the building of tiers can proceed in a 
manner such as this: 

meek 1. 
Any inspection task covering a worker error which has a 


cost greater than the maximum tolerance cost should be 
considered first to maintain the consumer's confidence. 
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TIER: 2: 

If any time remains after the first tier inspection tasks 
are allotted their time then that remaining time can be 
applied to the second tier of goals. The second tier 
consists of any visual inspection tasks not addressed in 
tier one. 

If there is insufficient time to cover all tasks 
in the first tier, then goal programming is invoked to 
determine the allocation of time across those first tier 
visual inspection tasks. If sufficient time exists for the 
first tier tasks to receive maximum time then they are given 
their full allotment of time and goal programming is invoked 
to allocate time on the visual tasks in the second tier. 

d. Achievement Function 

The achievement function takes the goals within 
one tier of the goal program and considers the deviations from 
each goal simultaneously. The deviations from goals are given 
weights by the decision maker in accordance with their 
relative importance to the organization. 

In the goal program for allocating visual 
inspection resources, the decision maker gives a weight to the 
time goal deviation and the accuracy goal deviation. The 
appropriate weight is multiplied by the deviation variable for 
each goal. The two weighted values are then summed to equal 


a total deviation for the problem. The achievement function 


minimizes this total deviation value. As the goal program 


48 








iterates, it trades off accuracy for time or time for accuracy 
until a solution is reached which best satisfies the two goals 
in light of their respective weights. 

The standard achievement function for allocating 
visual inspection time is given in Equation 2-9. 
(2-9) Minimize DEVIATION 

where DEVIATION = (TIMEWT * TPOS) + (ACCWT * ACCPOS) 
e. The Formulation 
The generic goal program to accomplish allocation 


of inspection time is shown in Figure 5. 


Achievement Function 
Minimize: 


DEVIATION = (TIMEWT * TPOS) + (ACCWT * ACCPOS). 


Goal Constraints 


pmegect to: 
m 
=X; - TPOS + TNEG = TIMEGOAL 
ae 
m 
a mee % oe C(emrom,). |] - ACGPOS + ACCNEG ~ =~ACCGOAL 
i= 
where m = the number of inspection tasks to be done. 
System Constraints 
Twins © %; S Trays, Formula = 1, o2)..., M 
X,, -.--, X,, TPOS, TNEG, ACCPOS, ACCNEG 2 O. 


Figure 5 - Goal Program Formulation 
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| ee SOFTWARE SUPPORT OF METHODOLOGY 

Suggested software support of the proposed methodology 
includes computer packages that swiftly perform a number of 
matrix operations and optimization. 

1. Matrix Operations 

A means to manage matrix operations is needed to 
transform the survey responses on worker error into useable 
data for the model. Some survey responses will be invalid due 
to internal inconsistencies. As discussed in Chapter II, B- 
4, eigenvalues are needed to determine the internal 
consistency in a group of judgments from a respondent. An 
eigenvalue solver or software which will allow the analyst to 
manipulate matrices to derive eigenvalues is necessary to 
permit a timely decision of whether to incorporate various 
respondent surveys into the model. 

A Programming Language (commonly known as APL) iS a 
software product that permits easy manipulation of matrices. 
The eigenvalue solver that accompanies most APL software 
packages is used on the illustrative problem of this thesis. 
APL is used in both Appendix A and Appendix D to derive the 
eigenvector from respondent surveys for the probabilities of 
crew error and to derive the maximum eigenvalue to determine 


respondent consistency. 


a6, 


2. An Optimizing Package 
A linear or nonlinear programming package is needed 
to optimize the objective function. There are 13 decision 
variables that Simultaneously need to be considered. 
a. Nonlinear Programming Package 

A nonlinear programming package will permit a 
straight-forward solution to the model. Data which show the 
increase in inspector accuracy as a function of time is 
extracted from experimental studies which closely approximates 
the type of visual task being modeled. A curve fit is made 
to that data and a function is generated that approximates the 
curve fit. That function, if convex, can be included directly 
into the goal constraints for use by goal programming. The 
curves in all studies indicated in this thesis have indeed 
been convex. 

In the illustrative problem which accompanies this 
thesis the versatile and powerful Generalized Algebraic 
Modeling System (GAMS) package is used. GAMS allows for 
nonlinear programming. An example of the input and output 
which is produced with GAMS is shown in Appendix E. 

b. Linear Programming Package 
A linear programming pacKage will permit a 


solution to the model. There iS a minor degree of difficulty 
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in that the continuous curve generated by the curve fit in 
nonlinear programming must be broken up and made piecewise 
linear so that a linear programming package can solve the 
problem. 

The remainder of this thesis is the presentation 
of a problem to illustrate the methodology of allocating 
visual inspection resources ina production facility suggested 


in Chapter 2. 
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Iii. BACKGROUND TO AN ARTILLERY PROBLEM 


A THE ARTILLERY ORGANIZATION 

The artillery fire support system consists of separate 
components which ‘contribute to the goals of "rapid and 
accurate" fires. Depicted in Figure 6 are the components of 


the artillery system. 


OBSERVER FIRE DIRECTION CENTER 


De 
af — 


FIRING UNIT 





Figure 6 - The Sub-system Components of the Artillery 
The observer, a Forward Observer (FO) or a RADAR section, 
spots a target. The Fire Direction Center (FDC) receives the 
target location information from the observer and translates 
that input into orienting data for the guns. The gun section 
prepares the ammunition, orients the weapon, and fires at the 
observer's target. Survey sections and meteorology sections 


Support the gun section by giving topological surveyed grid 
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control and atmospheric meteorological information input into 
the system. 

The system components noted above are grouped into 
artillery organizations. Several gun sections and one FDC 
constitute a battery of artillery, commanded by a captain. 
Several batteries of artillery, observers, and surveyors are 
organized into a battalion of artillery, commanded by a 
lieutenant colonel. Several battalions and meteorologists 


compose a brigade of artillery, commanded by a colonel. 


B. ROLES OF THE PLAYERS. 
The focus of this thesis is the Gun Sectiven. The Gun 


Section consists of the members depicted in Table l. 


TABLE 1. MEMBERS ©F A CUN SEC Wier 


position rank number yrs in 
service 
Chief E-6 1 5 im meles) 
Gunner E-5 il 2- 10 
Asst gunner E-4 1 1 - 8 
Ammo handlers E-1,4 3-7 Or =a 


The ammunition handlers prepare the shell that carries the 
munitions to the target, set the fuze that activates the shell 
at the appropriate point in its trajectory, and select the 
propellant charge that propels the shell out of the cannon and 


towards the target with the proper velocity. The assistant 
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gunner vertically orients the weapon tube to a designated 
position so as to obtain the trajectory needed to reach the 
range of the target. The gunner horizontally orients the 
weapon to a designated position to place the weapon tube on 
line with the target. The chief inspects the work of his 
subordinates to insure accuracy of settings and then commands 
the crew to fire the weapon. 

The FDC computes six elements of data that, when taken 
together, constitute the weapon aimpoint. Those six elements 
of data are the horizontal orientation setting, vertical 
orientation setting, shell type, fuze type, fuze setting, and 
amount of propellant. Firing the weapon at an aimpoint other 
than that directed by FDC is a gun crew error and is the 


responsibility of the chief. 


c. POSSIBLE ERRORS THAT A GUN CREW MAKES 
The four errors associated with the ammunition handlers 

are: 

1) improperly selecting the shell type, 

2) improperly selecting the fuze type, 

3) improper setting of the fuze, and 

4) improperly cutting the amount of propellant. 
The four errors associated with the assistant gunner and the 
vertical orientation of the weapon are: 


1) improperly setting the pre-designated four digit 
elevation number on the elevation counter, 


oS) 


2) improperly leveling the pitch bubble, 
3) improperly leveling the cross level bubble, and 


4) failing to have the proper two digit number in the 
piece correction window. 


Figure 7 shows the location of these four errors on the 
assistant gunner's mount of a towed howitzer. The four errors 
associated with the gunner and the horizontal orientation of 
the weapon are: 


1) improperly setting the pre-designated four digit 
number on the deflection counter, 


2) improperly leveling the sight mount bubbles, 


3) failing to have the proper two digit number in the 
piece correction window, and 


4) not traversing the weapon to acquire a proper 
Sight picture through the panoramic telescope onto 
the aiming reference point. 
Figure 7 shows the location of these four errors on the 
gunner's mount of a towed howitzer. 

In addition to the above twelve errors that a chief must 
check prior to firing his weapon, there are checks he must 
make regarding the firing data his crew receives from the FDC. 
When firing into a clearly defined area for a prolonged period 
of time a chief is given a schematic that depicts the limits 
of the area into which he can safely fire. The schematic that 
depicts the limits for the safe fire of a weapon is called a 


“safety T". The chief is required to check that the-settings 
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Figure 7 - Potential Gunner and Assistant Gunner Errors 
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being made by his crew are within the horizontal orientation 
limits, vertical orientation limits, fuze setting limits, 
propellant limits, and ammunition constraints imposed by the 
schematic. In this way the chief insures that crewmen have 
not set grossly inaccurate data due to garbled communication 
from the FDC or from fellow crew members. As a minimum, the 
chief insures by means of the "safety T" that the round his 
section fires will fall within the limits of safe fire for 


munitions impact. 


D. THE GUN CHIEF'S TIME ALLOCATION PROBLEM 

The artillery's primary mission is to support the ground 
gaining maneuver forces with firepower to destroy the enemy 
or diminish hiS will co tage. The artillery accomplishes 
this mission by providing fast and accurate fires when called 
upon to do so [Ref.43 p.13]. Each component of the artillery 
fire support system is called on to contribute to the twin 
goals of fast and accurate fire. 

General Otis states, "(Artillery is) a widespread organism 
whose ultimate function is to deliver massive firepower 
exactly when and where the force commander wants it [Ref 30, 
p 29}." Fast and accurate artillery fire is what the maneuver 
force demands and what the artillery force must deliver. This 
thesis recognizes fast and accurate artillery fires as the 


preeminent consequences of interest (or measures of 
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Bavectivéliess) for the entire artillery system. The final 
component, the gun section, shoots the projectile at the 
target. The crew of men that operate the gun is the focus of 
the illustrative problem of this thesis. 

The primary resources available during fire missions that 
directly influence gun crew firing accuracy are the gun 
chiefs. The commander employs the gun chiefs as his personal 
Quality assurance inspectors [Ref 31, p.3-5]. These personnel 
effect accuracy by inspecting for gun crew errors prior to the 
firing of each round. There are sixteen distinct, mutually 
excluSive errors noted previously that the crew can make when 
firing the weapon. The limited resource that the inspector 
expends to influence accuracy is the time that he takes to 
inspect for errors. The inspector's time is limited because 
artillery fires must not only be accurate, but fast. 
Excessive time by the inspector detracts from promptness of 
fires. 

Current U.S. Army literature does not specify a method by 
which an inspector is to allocate his pre-fire inspection 
time. It is expected that a gun chief will use his artillery 
experience, knowledge of his crew, and intuition to decide 
where his time will best be spent between the various error- 
prone tasks he must cover. Intuition is inadequate in complex 


decision making, and this problem requires a complex decision 


SI, 


[(Retasseaep. L7ae Systems analysis and logic dictate the 
spending of limited error-stopping resources commensurate with 
the importance of the error. 

This illustrative problem presents an allocation of 
inspection time model for use by a commander. The model 
quickly and efficiently analyzes the importance of gun crew 
errors for an artillery unit by using subjective input from 
unit leaders garnered by means of Surveys. The model then 
useS goal programming and suggests an allocation of pre-fire 
inspection time to reduce the probability of undesirable 
consequences due to errors. 

The alternative to allocation of resources by modeling is 
Stricttanper tien. If intuition is failing in a unit as an 
effective means of allocating time then modeling offers an 
alternative. The alternative to modeling by means other than 
the quick techniques of the methodology in this thesis is to 
model uSing input from empirical observations. The use of 
empirical observations to generate error data is not viable. 

The error data needed are functions of training, 
experience, readiness, and type of cannon weapon system in the 
Uliaatee These factors vary considerably between units. 
Consequently, information gleaned in one unit will have 1ittle 
use in another unit. The first three factors; training, 


experience, and readiness; can change considerably within the 
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same unit in a matter of weeks. Therefore, information 
gleaned in a unit may have little use in the same unit the 
following month. Gathering of unit error data through 
empirical observation for every artillery unit at frequent 
intervals is impractical, if not impossible. The modeling 
methodology must be as efficient as the situation is dynamic. 
The methodology presented in this thesis is useable by 


artillery units to allocate inspection time. 


E. ASSUMPTIONS 
Assumptions in this problem are in two categories. One 
set of assumptions isolates the effects of errors by the crew 
so that other off-setting or magnifying errors do not 
interfere with the analysis of the crew. The second set of 
assumptions facilitates modeling. 
1. Assumptions to Isolate the Problem 
The concern of this illustrative problem is the 
difference between the FDC'sS aimpoint (the target location) 
and the gun section's aimpoint (the expected location of shell 
impact). That difference is the net effect of the error that 
the gun crew makes and the chief fails to catch. This error 
is measurable in terms of the number of meters by which the 
Shell is expected to miss the target. 
To isolate the gun section regarding its contribution 


to fast and accurate artillery fire, it is necessary to make 
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some assumptions. For the purpose of this thesis the 
following simplifying assumptions regarding source of firing 
errors are made. 


a. The other artillery system components (besides the 
gun section) perform their respective tasks 
related to timely fire in the time that is 
expected of them. 


b. The aimpoint computed by FDC is the actual 
location of the target. 


1) No errors are made among the other 
artillery system components (FO, RADAR, 
FDC, Survey, Meteorology). 


2) The gun crews do not err when taking 
measurements for the FDC. Gun crews are 
periodically asked by the FDC to provide 
propellant temperatures and muzzle 
velocities to assist in a better computer 
SOLUTION 


c. The aimpoint of the weapon is the precise impact 
point of the shell. 


1) Dispersion is disregarded. Artillery 
fire always experiences normal dispersion 
in terms of range probable error and 
deviation probable Crron about the 
targeted aimpoint. 


2) No errors are made by the battery 
leadership during the emplacement of the 
cannon weapon system. Prior to being set 
up to fire, the unit leadership emplaces 
the weapons. Errors occur during eens 
emplacement which, if not corrected, will 
affect the accuracy of the artillery to 
follow. 


3) No errors are made when the gun crew 
performs routine serviceability checks of 
their weapon. Prior to weapon firing the . 
gun section performs tasks which can 
later affect accuracy. These tasks deal 
with such things as sight alignment, 
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checks to ensure equipment is within 
tolerance, and emplacement of aiming 
reference points. 
4) No errors occur in sub-systems directly 
related to the gun _ section. Events 
external to the gun crew which may 
negatively affect the accuracy of the 
artillery are defective fuzes, defective 
propellant, a round that fails to seat in 
the chamber when it is rammed, and an 
aiming reference point that is 
inadvertently moved. 
2. Assumptions to Facilitate Modeling 
a. Battery Homogeneity 
Gun sections are organized in batteries which 
train together and achieve some degree of homogeneity under 
their common commander. Among the duties of the commander is 
the establishing of training standards, standard operating 
procedures, and proficiency standards for the unit. Tevican 
be reasonably expected that standards are somewhat uniform 
across a battery. This uniformity allows for the gathering 
of unit data rather than individual section data. Within the 
unit, use of time and human error rates are basically 
homogenous. Because of battery homogeneity, data generated 
in one battery is of limited or no use in another battery. 
b. Competent Gun Chiefs 
A chief is capable. He is competent as a quality 


assurance inspector of his crew's work. Gun chiefs are 


required to go through thorough periodic testing to insure 
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their aptitude at finding all possible crew emer TCs 
assumed that a chief does not lose his ability to find 
errors. The chief's competence is such that he perceives his 
visual tasks as being relatively Simple. Inspection failures 
are the result of random error, not incompetence. 
c. Gun Crew Errors are Independent of Each Other 

Crew errors are not totally independent. A 
poorly performing crewman is more likely to set a second 
error than a superb soldier is of setting his first. The 
interaction between errors is complex and unpredictable. It 
would be extremely difficult to model that interaction. The 
assumption of error independence is a rough approximation of 
reality to facilitate modeling. 

d. Type I versus Type II Error 

If the costs of identifying a good product as 
flawed (Type II error) is minimal compared to the costs of 
failing to identify a flawed product as flawed (Type I error) 
then Type II errors can be disregarded with little loss in 
model validity. This allows for simplification of the model 
and, in fact, is often the case in quality assurance 
inspection work. In the artillery gun section Type II error 
is commonly considered insignificant when compared to Type I 


error. No chief has ever been ruined by being overcautious 
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and calling a mission flawed when it was not. Many a chief 
has been ruined by failing to catch a flawed mission. 
e. Simplicity of Visual Inspection Tasks 
The visual inspection tasks required on a gun 
section are relatively simple to a trained cannoneer. The 
types of errors to be identified do not require a detailed 
search, exceptional visual acuity, or higher level cognitive 
skills. 
f. Effects of Errors are Cumulative 
The errors a gun section makes can often off-set 
each other such that the full effect of an error is not 
realized. This model assumes that when multiple errors occur 
on the same mission that their effects are cumulative along 
one dimension. Therefore, this is a "worst-case" analysis 


type model. 


Oe, 


IV. MODELING THE ARTILLERY PROBLEM 


A. GENERAL CHARACTERISTICS OF THE MODEL 
The model to allocate a gun chief's inspection time is a 

goal program which suggests an amount of time a gun chief 
should spend on each of thirteen inspection tasks prior to 
every round fired from his weapon. The suggestion for time 
use is based on the following criteria for the tasks: 

1) The likelihood of crew error; 

2) The likelihood of inspector error; 

3) The potential costs of the fired error; 

4) The commanders' objectives. 
The method by which the data are gathered for the model takes 
into account the characteristics of the artillery weapon 
system and the level of training of the surveyed unit being 


analyzed. 


Joie THE PROBABILITY OF GUN CREW ERROR 

Each of the 16 crew tasks which the chief inspects with 
his 13 visual inspection tasks has a distinct probability for 
errok Based on the battery homogeneity assumption, a 
probability for crew error applies across all the gun 
sections of a battery. As a result of his firsthand 


Knowledge of the phenomenon, the gun chief has a strong 
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intuitive sense for the relative frequency of errors on the 
crew tasks that he inspects. 

Due to the battery homogeneity assumption and the fact 
that there are six or eight gun sections to a battery, there 
are seldom more than eight gun chiefs to give expert judgment 
on any crew error phenomenon under investigation. The paired 
comparison technique of estimating error is not viable unless 
there is a large number of respondents. Thus, the paired 
comparison technique iS not a practical solution for the 
illustrative problem in this thesis. 

The model accepts the gun chief aS an expert capable of 
rendering judgments on relative frequencies of crew errors. 
The model gathers this individual knowledge by means of a 
Survey. Appendix G contains copies of the Surveys 
administered to the gun chiefs. The Survey forces the chief 
to make enough comparisons to allow for an internal 
consistency check on his judgments (Saaty [Ref 17]). Those 
judgments found to be sufficiently consistent are converted 
to probabilities using the indirect numerical estimation 
technique (Swain and Miller [Ref 1]) and averaged together 
(Von Winterfeldt [Ref 29]}). The result is a deterministic 
point estimate of probability for each crew error in the 
battery. Appendix G shows the structure of the Survey. 


Appendix C demonstrates checking the consistency of 


Oy 


individual surveys. Appendix A giveS an example of 
converting paired comparison survey input into probabilities 


and the averaging of input from several experts. 


C. THE PROBABILITY OF INSPECTOR ERROR 
With 13 inspection tasks the gun chief checks the 16 
possible crew errors. Four of the crew errors, the four 
associated with the assistant gunner, are assessed by a 
Single check that a chief makes uSing a precision tool called 
a gunner's quadrant. Within certain bounds inspector error 
is a function of time taken to inspect a crew task. Those 
bounds for a gun section are explored. 
1. A Function of Time 
For a fraction of a second at the start of an 
inspection task the inspector experiences no increase in 
accuracy performance due to time required for eyes to focus 
and the starting of the visual search. As the inspector 
takes more time his probability for successfully identifying 
an error, given that an error exists, increases at a 
decreasing rate (Rizzi, Buck, Anderson [Ref 25]). This 
increase in accuracy performance continues until the 
inspector spends 1.5 times the standard amount of time he 
would normally spend on that visual task (Schoonard, Gould, 


Miller [Ref 24]). 
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The experimental findings of Schoonard, Gould, and 
Miller are heavily incorporated to establish an upper bound 
for the productive use of time for a gun chief performing 
visual inspection tasks. Schoonard, Gould, and Miller's 
study [Ref 24] closely approximates the situation in which 
gun chiefs find themselves when inspecting for crew error. 


The study picked a technical task, the inspection of 


integrated Sa Licon Circus chips, requiring trained 
inspectors. Each inspector had a minimum of six months 
experience inspecting for faulty chips. The actual visual 


search required the simultaneous searching for six or more 
Gifferent types of errors followed by a decision to accept or 
not accept the chip construction. The possible errors ranged 
from the simple to the complex in terms of the difficulty of 
discerning them during an inspection. The range of times 
that inspectors took to inspect a Specific chip for errors 
was 2.7 to 5.4 seconds. In every aspect of the study noted 
above, Schoonard, Gould, and Miller come extremely close to 
the situation of a gun chief inspecting for a gun crew error. 

The model accepts that from the time the chief starts 
searching for errors until 1.5 times the standard time spent 
on that inspection task [Ref 24] the chief experiences an 
exponentially increaSing probability of detecting an error 


[Ref 25]. At 1.5 times the standard time spent on an 


inspection task a competent chief, with no distracting 
factors, detects 100% of errors (Drury [Ref 26]). If given 
more than 1.5 times the standard time for inspection the 
inspector experiences degradation in his performance (Kochhar 
(Ref 23]). 

Drury [Ref 26] and Smith and Barany [Ref 22 and 32] 
both develop a curve which represents the inspector's 
probability for accurate inspection results as a function of 
time. The curve increases at a decreasing rate, forming a 
gentle, convex curve. The curve is continuous from a minimum 
time which reflects the time it takes the eye and brain to 
Start a visual search to a maximum time which reflects the 
limits of human vigilance for simple inspection tasks. The 
maximum time is assumed to be 1.5 times the standard time 
spent by the inspector on the task. The function chosen to 
represent the probability for inspector accuracy curve is: 
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: ok = Jee 
Where Y,= probability for inspector accuracy on task 1 
Ti= (Ty - Tyni) / CTmaxi - Torna ) 
T,= time chosen to spend on task; 
This curve approximates the curves that Drury [Ref 26, p 264] 
and Smith and Barany [Ref 32, p 5] found in their work and 


which are depicted in Chapter II.C. 


Dive Establishing Standard Time Use on Visual Inspection 
Tasks 


To utilize the experimental findings that greatest 
inspector accuracy is achieved at 1.5 times the standard 
amount of time spent on a visual inspection task requires the 
generation of standard inspection times on all visual tasks. 
The standard time used to perform each visual inspection task 
could be gathered by empirical observation, but this 
illustrative problem uses the alternative of gathering the 
chief's knowledge of time utilization by means of a survey 
fRef 14, p.6-1]. 

The chief has a strong sense for where his time is 
spent when he inspects for error. The chief performs paired 
comparisons of the amount of time spent on various inspection 
tasks. From the paired comparison data the model derives a 
point estimate for the time spent per inspection task by the 
gun chief. By averaging across all chiefs in a battery the 
model derives a battery estimate for chief's time spent per 
inspection task. See Appendix C for details on the 
structuring of a time use survey and of how a time use 
estimate is derived from actual data. 

3. Total Time to Perform All Inspection Tasks 

A limit on the total amount of time an inspector is 

permitted to perform all visual tasks may be determined by a 


commander. For general purposes of this problem, the Field 


Artillery School standard of 30 seconds is the amount of time 
permitted chiefs to inspect their crews prior to fire [Ref 
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D. THE PROBABILITY OF FIRING AN ERROR 

For an error to be fired the chief must be inspecting an 
improperly performed task and fail to identify the error. 
The probability that the crew performed a given task 
improperly is a point estimate derived by Survey input. The 
probability that the chief fails to identify the error is a 
function of time. The model is shown in Figure 8. 

A technique in nonlinear programming for evaluating the 
curve in Figure 8 is shown in Equation 4-2. This technique 
explicitly models the curve shape shown in Figure 8. 


(4-2) Ser = [ wv CL =, Po. * T; + Bea ie i=1,2... 3 


where T,;= (Ty - Tsry;) / (Tmax: - Tami), See Equation (4 —ae 


Poiscy = P( accuracy,) = probability for system accuracy 
on task,, 
P(error,) = 1 - P(accuracy,;) = probability for system 


error On@tasts 


EE THE COST OF FIRING AN ERROR 

The costs for firing an artillery gun crew error Tange 
from the insignificant or the unnoticeable to the very 
Significant. Some of the more important costs of a gun crew 


error being fired are: 


TZ 


1) loss of maneuver force confidence in their 
Supporting artillery; 


2) the cost of the resupply effort to deliver more 
ammunition for that which was wasted; 


3) loss of destructiveness due to inaccuracy. 
The most important costs are directly related to the 
consequences of interest for the system. Timeliness and 
accuracy of fire are the consequences of interest to the 
artillery fire support system. 

Each of the 16 crew tasks has a different cost 
distribution associated with it in terms of inaccuracy (in 
meters), should the crewman err at performing his task. The 
model uses the PERT cost estimation technique to determine a 
cost distribution for each error [Ref 33]. Senior battery 
personnel that are well versed in both fire direction and gun 
crew operations determine the largest, the smallest, and the 
most likely error of each of the 16 error types. Using 
tabular firing tables [Ref 38] for the battery's weapon system 
and assuming a range at which targets are most likely engaged, 
a Beta distribution cost curve (in meters) is established for 
each error type [Ref 38 and Ref 33]. See Appendix B for 
details of the PERT cost estimation technique. 

The potential number of meters by which a crew error 


causes the round to miss a target is a good indicator of 
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either the amount of damage that can be expected on the target 
(small errors) or the amount of maneuver confidence that may 


be lost (large errors). 


F. MULTI-OBJECTIVE CRITERION OF THE COMMANDER 
The commander has objectives that he wants accomplished. 
Suppose, for instance, that a given commander, first and 
foremost, wants never to lose the confidence of the maneuver 
force he supports. Second, the commander wants to have 
effects on target. These hypothetical but realistic 
objectives of a commander are pursued in the remainder of this 
model to demonstrate the flexibility and real-world 
applicability of goal programming. 
1. Making a Goal out of First Objective 

For the purpose of this problem, assume that the loss 
of maneuver confidence in their supporting artillery is the 
most costly and damaging thing that can happen to the fire 
Support system. There is good reason that this may well be 
jeme TOr any artillery unit. Loss of confidence results in 
hesitance on the part of maneuver to employ artillery and thus 
increases the chances of 1o0sSing combat engagements due to lack 
of firepower. 

Confidence is difficult to quantify and place on some 
continuous scale. Therefore, a crew error cannot be directly 


linked to an amount of confidence lost by the maneuver force 
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towards its supporting artillery. It is intuitive that a miss 
by a large number of meters corresponds to a larger loss of 
confidence by the maneuver whereas a small miss may be 
mistaken aS normal dispersion and thus goes’ unnoticed. 
Therefore potential number of meters by which a crew error 
causes the round to miss a target suffices as a surrogate for 
the amount of confidence lost. 

Below a certain distance threshold, maneuver 
confidence is not lost by artillery which misses its target. 
Artillery is an area fire weapon and it is expected by those 
who employ it that artillery will hit in the general vicinity 
for which it was called. Within that vicinity, or expected 
area of impact, the cost of crew error is just the cost of 
missing the target and the cost of wasted ammunition. 

The maneuver confidence radius represents the maximum 
distance from the target that a round can hit and still 
maintain the confidence of the maneuver force being supported. 
The commander's first goal is that his rounds must hit within 
the maneuver confidence radius of the targeted aimpoint. The 
commander needs to specify a maneuver confidence radius. 

2. Making a Goal out of the Second Objective 

The commander's second objective, getting effects on 

target, is clearly a function of both the accuracy of the fire 


and the timeliness of the fire. If accuracy is sought at the 


excessive expense of time then the target will likely move and 
escape destruction. If timeliness is sought at the excessive 
expense of accuracy then the target will escape destruction 
due to rounds missing the aimpoint. A trade-off between time 
and accuracy iS required to get effects on the target. The 


commander needs to specify the accuracy and timeliness goals. 


G. GOAL PROGRAM FORMULATION 

In the proposed multi-objective problem there are two 
objectives: avoid any loss of maneuver confidence and get 
effects on target. One goal is on the first tier: hit within 
the maneuver confidence radius of the target. Associated with 
the second objective is a tier of goals. On the second tier 
is an accuracy goal and a timeliness goal that the commander 
Specifies based on his weapon system and the targets on which 
his weapon systems will most likely be employed. 

This two-tiered set of goals lends itself to multi- 
objective, preemptive goal programming. On the first tier is 
the commander's goal of keeping maneuver confidence. Once 
inspector time is allotted and this first goal is achieved 
then the second tier of goals is considered. 

1. Decision Variables 

In goal programming to achieve allocation of visual 
inspection resources, inspector time spent on checking the 


work of others is the decision variable ( X,, i=the number of 


TA 


visual inspection tasks). The purpose of this goal 
programming is to achieve a most satisfactory allocation of 
inspector time. 
2. System Constraints 

The minimum and the maximum amounts of time that can 
be spent productively on each of the tasks are the two system 
constraints. These limits are based on the experimental 
findings of Drury [Ref 26] and Schoonard, Gould, and Miller 
[Ref 24]. Drury found a minimum amount of time required for 
an inspector's eyes, brain, and nervous system to function and 
start a visual search for error. Schoonard, Gould, and Miller 
found a maximum amount of time that an inspector spends at a 
visual inspection task before performance declines due to the 
limits of human vigilance. 

3. Goal Constraints 

There is not a single deterministic cost associated 
with each system error. There are three costs: a maximum, a 
minimum, and a most likely cost. Using the PERT “coon 
estimation technique a Beta distribution curve is generated 
to fit these costs. Visual inspection tasks are related to 
accuracy goals through their individual beta cost functions. 

Visual inspection tasks fall into consideration in 
tier one if their minimum possible cost exceeds the maneuver 


confidence radius. Visual tasks fall into consideration in 
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tier two if their maximum possible cost exceeds the maneuver 
confidence radius. Tier three visual tasks have maximum 
possible costs below the maneuver confidence radius. 

Time is allotted by tier, thus reflecting the 
commander's priorities. The total amount of time required to 
fully satisfy (1.5 times the standard time) all visual tasks 
residing in a tier is compared to the total inspector time 
available. If time required is less than time available then 
all visual tasks in the tier are allotted a full complement 
of time and the model moves to the next tier of tasks. itge 
time required to fully satisfy all visual tasks in a tier is 
greater than the time available then the goal program is run 
to determine allocation of inspection time among the visual 
tasks residing in the tier. 

Tasks within tiers are handled with the same goal 
Bemsctraints. These goal constraints are elastic, allowing 
some deviation if insufficient resources are available to 
satisfy all goals. 

The goal constraints are: 


(4-3) 
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for all tasks n where minimum cost task, 2> MCR, 
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fer 2 » X, - TPOS + TNEG = TIMEGOAL, 
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for all remaining tasks (13 - N) where maximum cost of 
task, 2 MCR, 


tLer so eK, - TPOS + —TNEG = TIMEGeI. 
g= 


for all remaining tasks where (N + P + Q) = 13. 


TIMEGOAL, is the overall goal for time in which all tasks are 
to be completed. TIMEGOAL, is the amount of time remaining 
from TIMEGOAL, after the first tier tasks are satisfied. 
TIMEGOAL, is the amount of time remaining from TIMEGOAL, after 


the first and second tier tasks are satisfied. 


(4-4) 
N 
aiewame: 2[P(error, ) * MAXCOST,] - ACCPOS + ACCNEG = O 
n= 
for all tasks n where minimum cost task, 2 MCR, 
P 
Cicme 2, {P(error, ) * MAXCOST,] - ACCPOS + ACCNEG = 0 
p= 


for all remaining tasks (13 - N) where maximum cost of 
task, 2 MCR, 


tier 3 > [P(error,) * MEANCOST,] - ACCPOS + ACCNEG = O 
g= 


for all remaining tasks where (N + P + Q) = 13, 


where P(error,) = the probability of system error on task; 
from Equation 4-2, 


MAXCOST;, = maximum cost of task, (see Appendix B), 
MEANCOST, = mean cost of task, (see Appendix B). 


MCR 


maneuver confidence values 
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The right hand side of the constraint represents a goal in 
terms of accuracy (in meters) that the commander wants to be 
met. 

4. Achievement Function 

The commander assigns a weight to the time goal 
Geviation and the accuracy goal deviation in the model. The 
commander must weight the deviations from each goal with 
respect to their relative importance to the fire support 
system. The purpose of the goal program is to simultaneously 
minimize deviations from goals within tiers. 

The commander is generally aware of the type of combat 
in which he will be employed and the type of targets on which 
he will be called to deliver his fires. Most targets of 
opportunity have the potential to move at some rate. That 
rate (meters per second) provides the commander the knowledge 
he needs to make a trade-off between timeliness of fire and 
accuracy of fire. The commander expresses this trade-off by 
the weight he assigns to the respective goal deviations. 

The appropriate weight is multiplied times’ the 
deviation variable for each goal. The two weighted values are 
then summed to equal a total deviation for the problem. The 
achievement function minimizes this total deviation value. 


As the goal program iterates, it trades off accuracy for time 


or time for accuracy until a solution is reached which best 
satisfies the two goals in light of their respective weights. 


The achievement function is given in Equation 4-5. 


(4-5) 
Minimize: 


DEVIATION = (TIMEWT * TPOS) + (ACCWT * ACCPOS). 
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V. APPLICATION TO ACTIVE ARMY UNIT 


A. CURRENT ALLOCATION OF INSPECTOR TIME 

The competent chief will attempt to at least look at every 
one of the 16 error-prone tasks accomplished by his crew. The 
chief's underlying belief is that he is a good inspector and 
that if he looks at the possible error he will likely catch 
the error if it is present. 

The following are the keys to modeling the current 
allocation of chief's time on the pre-fire inspection: 

1) The gun chief spends at least the theoretically 
derived minimum time for a visual inspection task 
on every task. 

2) The gun chief spends additional time on those 
tasks that he Knows are more frequently in error 
than others in the attempt to catch as many errors 
as he can. 

3) The chief does not trade time for accuracy. He 
shoots within the time required of him (30 
seconds) and checks as best he can (using rule 1l 
Memeemmamring that period of time. 

The model proposed is simple, yet it mirrors the chief's 
decision making process quite well. The idea behind the 
current use of inspector time is summed up by the one goal: 
stop all errors within the time constraint to shoot. A GAMS 


program which reflects these priorities and uses’~ the 


probabilities for error developed in this thesis is shown in 
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Appendix E. The allotment of inspector time resulting from 


this model is shown in TABLE 2. 


B. ALTERNATIVE ALLOCATION OF INSPECTOR TIME 

The following are the keys to modeling the alternative 
allocation of chief's time on the pre-fire inspection: 

1) Visual inspection tasks covering crew errors with 
minimum possible costs that exceed the maneuver 
confidence radius must be considered first for 
allocation of inspection time. The maximum 
possible costs of the errors are considered when 
the allocation of time is made. 

2) Visual inspection tasks covering crew errors with 
maximum possible costs that exceed the maneuver 
confidence radius are considered next for 
allocation of inspection time. The maximum costs 
of the second tier errors are considered when the 
allocation is made. 

3) Visual inspection tasks covering the remaining 
errors are considered last for allocation of 
inspection time. The expected costs of the third 
tier errors are considered when the allocation is 
made. 

The alternative model incorporates this information before 
allocating inspector time. In this example of an alternative 
plan for the use of the inspector's time the commander's goal 
is to stop costly errors. The value that drives the goal, in 
this instance, iS maneuver force confidence and target 
destruction. The results will be different than the current 


allocation of inspector time which was driven by the 


inspector's goal of stopping all errors. 
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Appendix E gives the GAMS computer runs of both the 
current and alternative allocation of inspection time by an 
active army unit surveyed at Fort Ord, California. TABLE 2 
shows the different allotments of inspector time per 


inspection task that resulted from the two model runs. 


TABLE 2. CURRENT VS ALTERNATIVE ALLOTMENT OF INSPECTOR TIME 


Inspector Task Current Alternative Difference 
Allotment Allotment Col2 - Coll 

Gl De Jae Zao iO 
G2 J. 3 BE 2 O5 =e Oa. 
G3 Peo 1 300 0 
G4 ie 60 1.46 - .14 
AG 7 a oask 8.90 + 1.69 
Al agsi®) de. O 
A2 dle, 98 ge = i205 
A3 i 70 Desi, - .43 
A4 Sa 3.40 O 
Sal ed 3 00 a AL tele 
52 2200 2. OO 0 
S3 Zols Co) 8, + 235 
S4 1.40 say, - .17 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 

Visual inspection to identify worker error often involves 
trained inspectors to perform a task requiring technical 
expertise. Human accuracy at visual inspection tasks is 
generally a function of the time taken to inspect. Inspector 
accuracy at positively identifying existing errors increases 
at a decreasing rate as a function of time. There are lower 
and upper bounds on the time axis creating a range of time 
over which this increasing inspector accuracy is a continuous 
variable. Those bounds have been explored experimentally and 
are available for use by systems analysts. 

If the possible human errors on a work site can be 
identified then expert judgments of the personnel involved in 
the actual operational work can be used to derive the 
probability for worker error on each task. In the same way, 
the standard time an inspector spends on the inspection of the 
task can be derived. Costs of system failures resulting from 
task error can be estimated by the management. 

The expected cost of doing business can be written as a 
function of inspector time spent on each component task in the 


system. If the decision maker can establish goals for the 


86 


system cost of a product and the acceptable amount of time for 
an inspector to spend on the product, then goal programming 
provides a method for the organization to optimize the use of 


inspector time. 


B. POSSIBLE APPLICATIONS OF METHODOLOGY 

Small-scale operations that employ visual inspectors to 
check the work of production personnel are prime candidates 
for use of this methodology. The small-scale operation is 
emphasized because it is most likely one in which a formal 
system analysis has not been conducted to evaluate optimum 
times for viSual inspection or optimal rates by which 
mPeoaguction must occur. 

An abbreviated but effective systems analysis approach to 
the proper utilization of inspector time requires only a few 
probability estimations and a few cost estimations. The fact 
that the personnel on the weapon systems can provide the 
probability of error information and the unit leaders can 
provide the cost of error information means that individual 
units can apply systems analysis to better achieve their 
goals. 

The pace at which the product is presented to the visual 
inspector has everything to do with whether the visual 
inspection will have the desired effect of the management or 


not. In this case a little bit of math goes a long way. By 
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using eigenvalues to establish validity of expert judgments, 
the model considers only those respondents whose judgments are 
consistent. Some adding, multiplying, and dividing achieves 
probabilities for each crew error. The cost estimation 
technique uses a simple beta distribution to model the cost 
distribution for each error. The model requires only three 
rough estimates on the part of a knowledgeable manager. The 
mean of the cost distribution is achieved by some subtraction 
and division. The probability of inspector error and the 
final cost of doing business is determined by goal 
programming. The input for the goal constraints are obtained 
from the decision maker. The result is a recommendation, by 
task, for the use of inspector time such that the organization 


goals are met as closely as possible. 


C. AREAS FOR FUTURE RESEARCH 
A curve fit on some experimental results done in the 
1970's on the relationship between time spent on a visual 
inspection task and the accuracy of the inspector at 
positively identifying existing errors yielded a curve of 
P(IA) = ee 


where P(IA) is the probability of inspector accuracy and 
T is the time the inspector takes to inspect. 


This is basically the curve used for all visual inspection 


tasks in the illustrative problem used in this thesis. An 
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area for future research is classifying different curves with 
different levels of difficulty of the visual inspection. It 
appears from the experimental results that with an increase 
in difficulty in the visual task, the rate of improvement in 
accuracy changes from increasing at a decreasing rate to 
increasing at a slower, more constant rate. If error-prone 
visual inspection tasks can be rated according to difficulty 
and different accuracy curves can be generated for difficulty 
levels then the model may be more valid. 

Another area that may yield interesting results is 
weighing of the various expert judgments using some criteria. 
Suggested criteria that are readily discernable from the 
respondents are time in service, time in the job, rank, and 
skill qualification test score. Weighted judgments may yield 
probabilities that are more reliable or verifiable than giving 
each respondent equal weight in their input to unit error 
probabilities. It may be determined that certain 
qualifications are significantly more important than others 
in the development of a gun chief. 

The model developed is entirely deterministic. The model 
develops probability and time estimates from several sources. 
From the several estimates a distribution could be established 
with a standard deviation on the mean point estimate. The 


cost estimate procedure using PERT methods generates a Beta 
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Gistribution for costs of each error. A simulation could be 
developed incorporating the distributions from all the data 
collected in this model which would yield a more robust 
solution. By running the simulation a number of times, 
confidence intervals for the variouS errors could be 
generated. 

This thesis put time and probability phenomena on an 
interval scale of measurement by using paired comparisons and 
indirect numerical estimation methods. The anchoring of that 
interval scale to a ratio scale was done by direct numerical 
estimation of some single phenomenon by the respondents. 
Direct numerical estimation has flaws as a technique for 
gathering data from operational personnel. An area for future 
research would be to design an experiment and generate actual 
values from empirical observation for the estimates which 


anchor the interval scaled values to a ratio scale. 
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APPENDIX A 


GENERATING PROBABILITIES FOR CREW ERROR 


The steps to generate probabilities for crew error using 
expert judgment are given below with a numeric example taken 
from survey input out of an active army unit. 

is Create a fault tree. See Figure 9. Worker errors 
appear in the lowest, or first level. Errors are grouped by 
worker, by work station, or by some other criteria that is 
easily discernable by the inspector. Those criteria form a 
second level of the fault tree and fall under the general 
category of "error". YEiGuei ana We error form the third 
level of the tree. 

2. Gather survey responses from inspectors. Inspectors 
are called on to relay their firsthand knowledge of relative 
frequencies of errors. Inspectors do pairwise comparisons as 
to relative frequencies of errors within criteria groups. 
When all groups have been judged then pairwise comparisons as 
to relative frequencies are performed on the - groups 
themselves. Finally a direct numerical estimation is made by 
the inspector as to the frequency of missions that occur which 
have any error. A sample survey used in an artillery unit is 


included in Appendix G. 


on 


3. Turn Survey responses into a reciprocal matrix. Ones 
appear on the main diagonal. Survey responses of relative 
frequencies fill the upper triangular matrix and their 
reciprocals fill the corresponding positions in the lower 
Triangular matnrx. See uiablemz. 

4. Derive the maximum eigenvalue and the eigenvector for 
the reciprocal matrix. The eigenvector is the relative 
frequency of each error from the perspective of one 
respondent. See Table 3. The maximum eigenvalue is used in 
Appendix C to derive a measure of consistency for the 
respondent. 

Dis Turn frequency scores into probabilities. By 
normalizing the eigenvector (making its components sum to one) 
the probability for each error is derived, given that some 
error within that group of errors did occur. See Table 4. 

6. Across all consistent responses within a unit (use 
method in Appendix C to establish consistency), average the 
probability of each error type to make a unit probability. 
See Table 5. Fill those unit average probabilities back on 
the fault tree in the P(j), P(k), P(1), P(m), and P(n) 
positions. See Figure 10. 

7. Take the direct numerical estimates of the probability 


for a fire mission containing an error of any sort. Average 
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them for a unit probability of fire mission error. Fill this 
value in on the fault tree in the P(i,) position. 

8. Use the multiplicative rule down each branch of the 
fault tree to derive a probability for each type of error on 


the first level. 


TABLE 3. RECIPROCAL MATRIX FOR GUNNER ERROR FREQUENCY. 


( taken from a respondent's survey on gunner errors ) 


relative | 

frequency| <--------- error type-------- 
of errors i z 3) a 
error 1.00 OC 3.00 S200 


TABLE 4. EIGENVECTOR FOR THE RECIPROCAL MATRIX 


error type 1.718 
error type 2. .587 
error type 3 .263 
error type 4 .263 


TABLE 5. NORMALIZED EIGENVECTOR 
Eigenvector normalized to make probabilities: 
error type ~39 
error type recy 


1 

2 
error type 3.14 
error type 4 .14 


2 


TABLE 6. AVERAGE OF RESPONDENT'S ESTIMATES 


Average of five consistent 
respondents' probabilities. 


error type 1.30 
error type 2 .36 
error type 3.18 
error type 4.16 
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Figure 9 - Probability of Crew Error Fault Tree 
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Figure 10 - Probability of Crew Error Data 
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APPENDIX B. 
GENERATING COST ESTIMATES FOR ERROR 

Crews make errors and inspectors fail to identify those 
errors. The costs that result are measured in meters by which 
the error is expected to cause the round to miss the target. 
Each error has a maximum amount that it can cost the system. 
That maximum is affected by such things as unit standard 
operating procedures and unit safety measures which may 
preclude an extreme error. One example is the M102 howitzer. 
When traversed, the entire metal structure pivots on a ball 
mounted to a base plate. If traverse limits exist for firing 
safely then it iS a common technique to drive metal stakes in 
the ground abutting the metal structure when it is lined up 
to fire on the left and right traverse limits thus physically 
limiting the weapon to firing within safe traverse limits. 
The result is an administrative method of precluding gross 
firing errors to the left or right of the target area. This 
naturally reduces the possible cost of traverse, or deflection 
error. 

Because there are many procedures in the field on five 
different cannon weapon systems for precluding various firing 


errors, there exist no minimum and maximum costs for crew 


errors that apply across the army. It is within the means of 

senior personnel in each unit, however, to consider the unit's 

procedures and weapon system to establish a minimum, maximun, 

and modal value for each error type. The steps to take in 

establishing the key values for each error type are reviewed. 
1. Assume an average range at which targets are 


engaged. Assume a charge with which those targets 
are engaged. 


2. Extract information from the tabular firing tables 
(TFTs) for the respective range and charge 
assumed. Information to be extracted is the 


effect (measured in meters) of a round that, when 
Fired, 1SeeLrLlrewy. 


a) 1 mil in elevation 
b) 1 mil” im” deft lecrron 
c) 1 square weight 
qd) 1 charge of propellant 
e) 0.1 second on time fuze setting 

3. Consider unit procedures that effectively limit 
the size of errors that can possibly be fired. 
Determine a maximum amount of error that can be 
set by a worker in terms of the items listed above 


(mils, Square weights, charges, seconds) and 
convert that amount to meters on the ground. 


An Given that he must set an error, determine a 
minimum amount of error that a crewman can 
possibly set. For example, a crewman can set no 


less than a one charge error without performing 
perfectly in that regard. So the minimum possible 
error for charge is that number of meters 
corresponding to a one charge error. 


9. Given that he must set an error, determine a most 
likely error that can be fired. 
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In the case of a 105 mm artillery unit at Fort Ord, 
California, the following assumptions were made about the 
potential costs of the various inspection task errors. 

1) Safety Ts used, 
2) Safety stakes employed, 
3) Dog-legged safety box in impact area, 
4) Charge 5 propellant required, 
5) Range of target: 6,000 meters. 
The following matrix was developed from the expertise of 


several unit leaders. 


TABLE 7. COST OF FIRING A CREW ERROR (IN METERS ) 


estimated from computed assuming 
experience a Beta Distribution 

Inspection Min Modal Max Mean Std 

Task Cost Cost Cost Cost Devn 
Gl O 30 2400 420 400 
G2 O JUS; 200 40 oS 
G3 O 60 2400 440 400 
G4 O 15 400 74 66 
AG O 20 2600 447 433 
Al 950 1200 2400 5S 242 
A2 Vass. 30 100 41 152 
A3 O 20 hy ay, 33 20 
A4 O 100 2500 483 417 
$1 10 ZS. 2400 481 398 
2 15 36 2000 422 330 
So 25 260 2500 594 412 
S4 O 100 IE SY9) 92 25 
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APPEND vase. 
GENERATING TIMES FOR INSPECTION TASKS 

Two methods are suggested by the NRC for generating 
standard times for inspection tasks to be completed. One is 
by measuring empirical observations and the other is by expert 
judgment. Making empirical observations is the preferred 
method and is the one suggested if a unit has the time and 
resources to devote to an experiment to meaSure a number of 
observations. 

The steps to generate standard times for inspectors to 
inspect for crew error (uSing expert judgment) are given below 
with a numeric example taken from survey input out of an 
active army unit. 

1. Create a fault tree of crew tasks. See Figure 10. 
Worker errors should appear in the lowest, or first level. 
Errors are grouped by worker, by work station, or some other 
criteria that is easily discernable by the inspector. Those 
criteria form the second level of the fault tree. 

2. Gather survey responses from inspectors. Inspectors 
are called on to relay their firsthand Knowledge of relative 
lengths of time required to inspect for various errors. 


Inspectors do pairwise comparisons as to relative lengths of 


time to inspect for errors within criteria groups. When all 
groups have been judged then pairwise comparisons as to 
relative lengths of time required to inspect the groups of 
errors themselves are done. Finally a direct numerical 
estimation or an empirical measurement is made by management 
of the time required to inspect for one task. 

3. Using the one task as a reference point, all other 
tasks are given a length of time for inspection. When an 
estimate exists for all tasks then a unit average is taken and 


that becomes the standard inspection time for the unit. 
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APPENDIX D 
CONSISTENCY CHECKS ON SUBJECTIVE EVALUATIONS 
The method for checking internal consistency of a set of 
pairwise comparisons made by a respondent iS reviewed. This 
particularly applies to comparisons of relative frequency of 
various events, such as event A is three times more likely 
that event C. 

1. Follow steps 1 through 4 of Appendix A to derive a 
maximum eigenvalue for the respondent's reciprocal matrix. 

2. Generate a consistency index (CI) for the respondent's 
matrix by 

[ Amax7N ] / [ N=i J] = CE: 

3. Generate a quantity of random reciprocal matrices of the 
Same dimension and containing the same range of values aS was 
possible to the respondents on the survey they were 
administered. The main diagonal of these matrices are ones. 
The upper triangular cells of the matrices are filled with 
values picked at random from the possible range. Complete the 
lower triangular cells with the corresponding reciprocal 
values of the upper triangular cells. Compute the maximum 
eigenvalue for each matrix and average all of the maximum 


eigenvalues together. The result is an estimated maximum 
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eigenvalue for a respondent who gives totally random responses 
to the original survey. 

4. Generate a consistency index for the random matrix 
[Cl andomd * 

[eAveveanccneelmy LN — lag = Cl. naom: 
5. Generate a consistency ratio by 
(CR pont) 7 (Clesnaon) = CR- 

If CR is less than or equal to .10 then there is a sufficient 
level of internal consistency in the respondent's judgments 
to consider them for analysis work [Ref 17, p.83]. 

Step number three above requires some effort and 
Significant computer resources on the part of an analyst. 
This step is made considerably easier if the analyst prepares 
the respondent's survey so that only values between one and 
nine (and their reciprocals) can be selected by the 
respondent. The range of one to nine recurs frequently in 
human judgments. For this range of values Saaty [Ref 17] 
generated consistency indices (CI) for random reciprocal 
matrices from size (3 X 3) to (10 X 10). Table 7 shows the 


random consistency indices for matrices up to (10 X 10). 


TABLE 8. Random Consistency Indices (CI,_an,4om) 


onl Omo@meo. oS 98-90 1.12 1224 1.3251.41 1.45 1.49 


random 
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APPENDIX E 
USING GAMS FOR GOAL PROGRAMMING 

GAMS is used to perform the allocation of inspection time 
by the Current Model and the Alternative Model. 

1. The Current Model 

In the Current Model the chief is aware of only the 

inspection tasks he must perform, the relative frequency by 
which the tasks are successful at discovering crew error, and 
the relative amount of time it takes to perform the visual 
tasks. Given only that much information the chief's strategy 
is to stop as many crew mistakes as he can in the time 
allowed. The assumption is made that the chief's intuition 
works very well with the knowledge that he has and that he 
does accomplish his objective of minimizing the probability 
for a crew mistake. It is reemphasized that the chief does 
not consider the relative magnitude of the costs of the 
mistakes when he currently performs the pre-fire inspection. 
The GAMS program for currently allocating inspection time is 
labeled "Current Model". 

2. The Alternative Model 

The Alternative Model takes what the chief knows in 

the Current Model and then takes into account the COSTS Gea 


errors. The purpose of the Alternative Model is to not make 
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costly mistakes while meeting the commander's guidance for 
fast and accurate fires. In the first tier of the preemptive 
goal program any inspector error which has a minimum cost 
greater than the commander's maneuver confidence radius (600 
meters for the illustrative problem) is considered. Any first 
tier error will certainly have a huge negative effect on the 
artillery system. Every possibility for first tier error must 
be precluded. This is done by placing a large weight (BPWT) 
On any positive deviation from the accuracy goal. 

In the second tier of the preemptive goal program any 
inspector error which has a maximum cost greater than the 
commander's maneuver confidence radius is considered. Any 
second tier error potentially has a huge negative effect on 
the artillery system. After first tier errors are precluded 
then every possibility for a second tier error is precluded. 
Again, this is done by placing a large weight (BPWT) on any 
positive deviation from the accuracy goal. 

In the third tier of the preemptive goal program all 
remaining inspector errors are considered. Time that remains 
after the first and second tier errors are precluded is now 
allotted to the third tier inspection tasks. The commander 
now weights timeliness of fire and accuracy of fire relative 
to each other. In the illustrative problem the commander 


chooses to weight them equally (TIMEWT = BPWT = 1.0). 
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An asterisk in the first column of the GAMS computer 
input is a line that the computer program does not read. In 
the Alternative Model (2) asterisks are used in the PARAMETERS 
part of the program so that only the inspector tasks for the 
particular tier being considered are entered into the program. 
The GAMS input for the Alternative Model's first tier (2) is 
given. The second tier and third tier are the same programs 
but different tasks are allowed or disallowed by using 
asterisks. In each of the three tiers the SCALARS change to 
reflect the tier. The first tier starts with 30 seconds of 
time available (RESPTIME). The weights for time deviation 
(TIMEWT) and accuracy deviation (BPWT) are 1:600. The second 
tier starts with 28.2 seconds of time available. The weights 
for time and accuracy deviation remain 1:600. The third tier 
starts with 3.6 seconds of time available. The weights for 
time and accuracy deviation become 1:l. 

3. The Output 

a) The Current Model output is shown first. The 
allocation of time by the Current Model for each 
inspection task is shown in the column labeled 
LEVEL between the LOWER and UPPER limits. This 
appears near the bottom of the page of GAMS 
computer output. 

b) The Alternative Model output is on three pages. 
At the bottom of the first page is the allocation 
of time for those tasks identified as in the first 
tier (task Al). 

c) At the bottom of the second page is the allocation 


of time for those tasks identified as in the 
second tier (tasks Gl, G3, AG, A4, Sl, S2, S3). 
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d) At the bottom of the third page is the allocation 
of time for those tasks identified as in the third 
tier (tasks G2, G4, A2, A3, S4). 
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The Current Model. 


CURRENT MODEL 
HE HE HEH HE HE IE IE HE HEH HE IE IE HEHE IE IE I HE HE HE HE IE IE IE IE IE IE HE HE HE HE IE HE IE DE IE IE IE IE FE IE IE IE IE IEE HE IE HE IE DE IE DE IE HC IE IE HE IE DE IE IE HE HE EEE 
SET 
I thirteen inspection tasks in four categories 
/G1,G62,63,G64,AG,A1,A2,A3,A49,581,82,83,5849/}3 
HEHE HEHEHE HE HEHE HEHE HEHEHE HE HEHE HEHE HE HEHE HEHE IE HEHE HE IE IE IE HE HE IE IE IE IE IE IE IE IE HEHE IE I I HEI HE IE HE HE HE IE HE IE IE IE HE HE IE IE HE HEE HEHE HE HEE 


PARAMETERS 
HH HH HE HEH HE HK HE HEHE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HEE HE HE HE HE HE HE HE HE HE HH HE HE HE HE HE HE HE HE EH HE HK EE HK HE EK KK KK KK 
X these min times are theoretical; based on experiments of min time for 
X eyes to focus and mind to grasp what eyes are seeing 


MINCI) min time possible on task i- theoretical seconds 


Prag 
GC) NW 
NM 


~“" > 

N Ww 
NNNNNN NN ONIN 
‘\ 


“Yi 
iN 


PHESSSESSESSSSESESLESSSESSESSSESSSESSESESSESSTSSESSSESSESESTSSSESESESSSSSESE SSE SESE Fi 
X¥ these times incorporate theoretical 1.5 times max values of standard 
x time taken by gun chiefs to perform the task. This is time needed 
x to insure (100%) that task is done correctly. 


MAXCI) max time spend on task 1 - from paired comparisons 

/G1 2.8 
G2 Sac 
G3 1.0 
GG 126 
AG 8.9 
Al 1.8 
A2 4.8 
A3 1.7 
AG 3.4 
Si 3.0 
S2 2.0 
$3 525 
S4 1.497 


eee TTS TET TT TT TTT ET TT TTT TT TTT TET TESST TSS SSS SSESSSS SSS SSE SSESSESES ESTES SF F 
NEPROBCI) probability of no crew error task i - detm by INE method 


/G1 29 92e00 
G2 -990640 
G3ees 995520 
G4 -995840 
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The Current Model (continued). 


AG .974000 
Al - 988846 
A2 -990874 
A3 -987325 
AG ~982255 
31 995652 
S2 993994 
33 -991264 


S4 -9918107 
HEH HHH HH HEHE IE HIE IE EK HH HE EH IE KKK HH IE I HE HHH EH HE EK EE HE EK OK EK KX 
EPROBCI) probability of crew error on task i- detm by INE method 
/G1 .007800 
G2 009360 
G3 .004680 
G4 004160 
AG -026000 
Al 011154 
A2 009126 
A3 012675 
AG 017745 
$l .004368 
S2 .006006 
$3 008736 
S4 .0081907 


HHH HH HH HHH HH HE HH HH HH HE EH HH KE EE EE EK HK HK HH HH HH KKK KKK KEK 

xX A chief does not consider the costs of various errors as he determines 
*X where he will spend his time during the inspection. His sole objective 
*¥ is to stop as many errors as possible in the time allotted 

HHH HEH HH HE HE HEH HE HH HE HE HE HE ME EE EE EE EE EE EE EE EK EK KKK MRK UKE 


PSS SS SES SESS SESS SSCS SSS SSCS es eee ss ee ees te eee ee eee ee eee eee eee ete Sees ss: 
SCALARS 
RESPTIME total time allowed for guns to shoot / 30.07 


HH HH HH HEH HEHE HHH HH HK HK HH HK HH HE HE HK HE KK KK KK KK KK KKK KKH 
VARIABLES 
XGie) time to be spent inspecting each task in battery 
DEVIATION deviation from the objective function; 


3 3 3 3 SE IE SEE BE HE EO 3 BE 3 3 3 BE 3 BE HE BE BE FE HK IE IE IE HH IE HE IE IE HH TEE EE KH EK HE EEK KH 
POSITIVE VARIABLES 
XCI); 


HE HEH HEH HEI HEH HHH HHH HEH HH IE IE I IH HH HE I IK HK HH HI HH KH HK HH HK HK KH KKK 
EQUATIONS 
OBJDEF achievement function 
TLIMIT time limit; 
HH HH HH HEH IE HEH HE ME HE HE HE HE IE HEI IE HEH HH HH HHH HEH I HH HHH HE IE EH HE EH EE HK KKK KK KX 
*¥ this equation sums the probability for system error (which is a 
*¥ function of the decision variable) across al] the visual inspection 
* tasks AND assigns that sum a variable name of DEVIATION. 


LOY 


The Current Model (continued). 


eae . 7 re -— ae 


GPNLP2 


* MINIMIZE 
OBJDEF. aaa are =E= 


SUM ( I 
C 1 - SQRTC CSQRCEPROBCI)D)) ® CXCID-MINCI)D) 7 CMAXCID-MINCI)) ) 
- NEPROBC(I) ) ) 3 


3000000000000 0000000000 I0G 0000000000 CII IIT IIT ITSO 
* this constraint limits time spent by the inspector across all visual 
X inspection tasks to less than or equal to the commander's goal 


*% SUBJECT TO 
TLIMIT.. SUM (C I, XCI) ) Fl RESPTIME ; 


ADDITIONAL CONSTRAINTS involving min and max times allowed. These 
keep x in the feasible region as problem iterates, otherwise stops 


X.LOCI) = MINCI)*®1.01; 
X.UPCI) = MAXCI); 


If region is convex, then no matter where progrem sterts, the 


solution will be the same. This forces program to start internally 
on the convex region because funny things happen on the edges 


X.LCI) = MINCI)®1.02 


me 


MODEL GPHLP2 /ZALLZ; 
SOLVE GPNLP2 USING NLP MINIMIZING DEVIATION; 


Pome me eee et er feet et fet at freed feed fee feed fed feed ed feed feed fed feed feed fed feed fd feed feed 

DN DANA LOW Et Gt Ga AD AD AD AD AD AD AI AD AY A) bee me tee es 

HOO OMNDNNODAWNHOOONUNNAWN HOO ONAN 
oe 
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2. 


GPHLP2 


HK 


DAN OOANAUWDWA— OW ONAN D Ww 


PRY ADA PAD AD be ee ae eee et pent nt pene eet fee 


The Alternative Model. 


ALTERNATIVE MODEL (first tier) 
3000000000000 0000000 RIB BO BBR RBE 
* THIS MODEL PERFORMS FIRST TIER OF PREEMPTIVE GP CON THOSE TASKS WITH 
* MIN COST GREATER THAN THE MANEUVER CONFIDENCE RADIUS = 600m) 


X¥resp time ballpark ti.wt. bp.wt 
x 30 0 / 1 4 600 # 
x 


HE HE HE HEHE HE HE HE HE IE HE HE HE HE HE HE HE HE HE HE HE IE HE HE DEE HE DE HE HE DE DE HE HE HE DE DE HE HE IE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HEHE HE EE EER HER 


SET 
I thirteen inspection tasks in four categories 
7 


Gl, 
Ge, 
G3, 
G4, 
AG, 
Al 


mK KK KK 


x KK mK KK KKK 
" 
b= 


HH HHH HHH HE HEH HH HH HE HE HE HE HE HE HE HEH HE HE HE HE HE HE HE HE HE HE HE HE HEHE HEHE HE HE HE UE HE EE HE HE HE HE HE KK RK RRR 


PARAMETERS 
SESS S252 88 232 25 SS SSSSSSIESSSESSSESESSSESSESSESLSESSSESESSESSSSESSESS ESSE 2222 8 | 
* these min times are theoretical; based on experiments of min time for 
*¥ eyes to focus and mind to grasp what eyes are seeing 

MINCI) min time possible on task i- theoretical seconds 


/ 


x «KKK OM 

Yr» G) 

nN or 
MNNNNINININION NIN 


x eK KK KOK 
" 
b— 


/ 
2 S22. 2.0.0.2.0.0.2.0,.5,.2.2.0.0,20.0.0.0.0.225,.5.0.0.5.0.1 22 22,28 2.2.2.2 2.6.5,.5,.5 ¢,5,0,0.0,0,0,2,5,5,0,5,5,5,0,5,5,5,5,5,0,8, 0.0.05: 


x these times._incorporate theoretical 1.5 times max values of "normal" 
¥ time taken by gun chiefs to perform the task. This is time needed 


jeaed 


The Alternative Model (continued). 


GPNLP2 


% to insure (100%) that task is done correctly. 
MAX(CI) max time spend on task i - from paired comparisons 

pa ; 
x Gl Zag 
% G2 ro 
x G3 1.0 
x GG 1.6 
x AG 8.9 
Al 1s 
x A2 G.8 
x A3 Ly 
x AG 3.4 
x $1 3.0 
x 52 2.0 
x $3 SoS 
x $4 1.4 


7 
HE HEHE HE HE IE HE HE HEI IE HE IE IE IE HE IE HE IE IE IE HE HE IE IE I UE IE HE HE IE IE HE HE IE DEE DEE IE IEE EH IEE IE HE HE IE EE EI IH HH EK KE 
NEPROB(I) probability of no crew error task i - detm by INE 


=H OD OOD O00 0000 OO 00 WO WO 09 00 00 00 00 09 00 00 09 CO NININISININSININSINARAKRARAKAAAU 
DUNK BVO ONDUDWNH OO MNDUDWN HOD ONDUDWNHOVWDONAUDWNHHKOUDMNUAMAWNMOW 


7 
x Gl ~992200 
x G2 -9906490 
x G3 ~995320 
x G4 9958490 
x AG .974000 
Al ~988846 
x A2 -990874 
x A3 29587525 
x AG 982255 
¥ 3S] ~995632 
x Sc 993994 
x 53 991264 
x S4 .991810 
/ 
HH HH HHH HH HHI HHH HH HHH HH HHH HH HH HH HHH HH HH HH HHH HHH HHH HHH HK KKK KKK KKK 
EPROBCI) probability of crew error on task 1 ~- detm by INE 
] 
] / 
] ¥ Gl 007800 
1 ¥ G2 .009360 
] x G3 .004680 
] x G4 .004160 
] ¥ AG .026000 
1 Al 011154 
l x A2 009126 
l ¥ A3 012675 
1 ¥ AG 017745 
] ¥ $1 004368 
] - Sz .006006 
] ¥ $3 .0087 36 
] » S34 " 008190 


ise 


The Alternative Model (continued). 
GPNLP2 


JE HE HI I HE HEH HE HE I HE HE IE I HE I IE HE HEH I HE IEE IE HE HE IE IE HEH HE HEE HEHE I IE HEH IE HE HEH IE HEH HI HE HE EK KKH 
* these costs are derived using 2 criteria. First - mean cost is based 
X on data from firing tables assuming range to target 6000 meters, and 
xX beta distribution (pert estimation techniques). Second - max cost 

* ais based on data derived from firing tables. 


—Ooo On A Un 


MCOST(CI) mean cost of error i - detm by PERT Beta distn techn 


* SET 2 (max costs) 
7 
x Gl 2400 
x G2 200 
¥ G3 2400 
ra GG G00 
xX AG 2600. 
Al 2400. 
% A2 100. 
x A3 120. 
+ AG 2500. 
» S] 2400. 
x $2 2000. 
x so 2500. 
x $4 150. 
/ 
HEH HEHE HE HE HE HEHE HE HE HEHE HEH HH HE HE HE HE HE HEE HE IE HE HHH HI I HEH HE HEH HEHE HEI I IE HEH HH HH HEH HEH HEH HH HHH KKK KKK 
SCALARS 
RESPTIME total time allowed for guns to shoot / 30.07 
BPARK number meters in which aimpoint must be 7” 000/ 
TIMENT weight placed on time pos devn in OBJ 4 41.07 
BPHT weight placed on error pos devn in OBS /7/ 600 /; 
Pee SS S52 2255 53 55 5 5 ss ee tee eee eee SS ESSE SSS SSS SESS SESE SESE SESS S 3 
VARIABLES 
XCI) time to be spent inspecting each task in battery 
TOFF deviation from the time goal constraint 


BPOFF deviation from the ballpark goal constraint 
DEVIATION deviation from the objective function; 


IE IE IE IE IE IE 3 3 3 3 IE 3 SIE IEE 3 3 3 3 BEE IE IE DE 3 3 3 3 I IE IE IE IE DE EC EI BE BE DE JE JE IE JE IC EE IE DE 3 HE 3 IE IE IE IE DE IE IE IE HE HEH 
POSITIVE VARIABLES 
X(I), TPOS, THEG, BPPOS, BPNEG ; 


HH HEH HK HK HE HE HK HH HK HE HE HE HE HE HK HE HE HE IE HE HE HH HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HE HH HE HE HE HE HE HE HEE KK MK 
EQUATIONS 
OBJDEF achievement function 
TLIMIT time limit 
DLIMIT distance limit; 


PESTS SSCS SS SST SS ESTES SSS SLES SSS SSS SS SS ST SES SSS SLES ESET SSS SELES SSS 25. 
* MINIMIZE 
OBJDEF.. DEVIATION =E= 


mt fet ma feet fed emt eet md fe fet fe Baad ed fet feet fae fed fd md fed fred md fred fed feed feed fd feed fee fad fad fed famed fad famed feed femme fem feed feed fem fet famed fem famed ft feet fered Seed feed fot feet fet fd et 
SNANANNNKHANKDAKDOIIII IND DDD DDD DD DOO OOOO A) AD AAD AAD AD AD AAD AAD te ee te ee ee 


OO CONAN DWAY HOO ONAN OWN HOO OND DWA HOO ONDNAWA HOO MUANUAWA) 


iS 


The Alternative Model (continued). 


GPNLP2 


( TIMEWT * TPOS ) + (€ BPWT ® BPPOS );3 


300300000000 
x SUBJECT TO : 
TLIMIT.. SUM ( I, XCI)D J) - TPOS + TNEG =E= RESPTIME ; 
DLIMIT.. SUM ( I, 
C 1 - SQRTC CSQRCEPROB(I))) % CXCID-MINCID) 7 CMAXCID-MINCI)D) ) 
- NEPROB(I) ) x MCOSTCI) ) -BPPOS + BPNEG =E= BPARK 3 


eT SS ST SS SS SS SSS SSS SESSSESSSESSESSIESSSSSESSESSESSESSESSSESSSESS ESE SS FF : 


* ADDITIONAL CONSTRAINTS involving min and max times allowed. These 
¥ keep x in the feasible region as problem iterates, otherwise stops 


X.LOCI) = MINCI)¥1.01; 
X.UPCI) = MAXCI);3 


* af region is convex, then no matter where starts, soln should be same 
* Need to start internal because funny things happen on edges 


X.LOI) = MINCI)¥1 .02 


MODEL GPNLP2 ZALL/; 
SOLVE GPNLP2 USING WLP MINIMIZING DEVIATION: 


Grad mt emt fed fed feed ed amet md ft feed fame famed fed feed feed feed feed eed feed feed fed fed 
O00 OO 09 09 09 00 08 O98 0 OO OO NI NIN 
ANE DVWANNMNOWN = OO ONAN OWA 
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3. The Output 
a. The Current Model 


GPHLF2 


SOLUTION REPORT SOLVE GPNLP2 USING NLP FROM LINE 150 


se0°l VE SUMMARY 
MODEL GPNLP2 : OBJECTIVE DEVIATION 
PYPeE NLP DIRECTION MINIMIZE 
SOLVER MINOS5 FROM LINE 150 
¥*XXX SOLVER STATUS 1 NORMAL COMPLETION 
RXR MODEL STATUS 2 LOCALLY OPTIMAL 
XXRXKX OBJECTIVE VALUE 0.0151 
RESOURCE USAGE, LIMIT 0.263 1000.000 
ITERATION COUNT, LIMIT 23 1000 
EVALUATION ERRORS 0 0 
mereN OS <--- VERSION 5.0 APR 1984 


COURTESY OF B. A. MURTAGH AND M. A. SAUNDERS, 
DEPARTMENT OF OPERATIONS RESEARCH, 
STANFORD UNIVERSITY, 
STANFORD CALIFORNIA 94305 U.S.A. 


WORK SPACE NEEDED (ESTIMATE) -- 823 WORDS. 
WORK SPACE AVAILABLE -- 1179 WORDS. 
(MAXIMUM OBTAINABLE -- 164974 WORDS.) 
EXIT -- OPTIMAL SOLUTION FOUND 
MAJOR ITERATIONS 1 
NORM RG 7 NORM PI 5.001E-09 
TOTAL USED 0.28 UNITS 
MINOSS TIME 0.17 CINTERPRETER - 0.05) 
LOWER LEVEL UPPER 
---- EQU OBJDEF 0.1300 0.1300 0.1300 
---- EQU TLIMIT -INF 30.0000 30.0000 
OBJDEF ACHIEVEMENT FUNCTION 
TLIMIT TIME LIMIT 
---- VAR X TIME TO BE SPENT INSPECTING EACH TASK IN BATTERY 
LOWER LEVEL UPPER MARGINAL 
Gl 0.7070 292627 2.8000 EPS 
G2 0.7070 2.5660 3.2000 EPS 
G3 0.7070 1.0000 1.0000 -0.0056 
G4 0.7070 1.6000 1.6000 -0.0001 
AG 2.0200 7.2167 8.9000 ; 
Al 0.7070 1.8000 1.8000 -0.0029 
A2 0.7070 1.7816 4.8000 EPS 
A3 0.7070 1.7000 1.7000 -0.0042 
AG 0.7070 3.4000 3.4000 -0.0011 
$1 0.7070 | 1.1417 3.0000 EPS 
$2 0.7070 2.0000 2.0000 -0.0001 
$3 0.7070 2.1513 3.5000 EPS 
$4 0.7070 1.4000 1.4000 -0.0037 
LOWER LEVEL UPPER 
---- VAR DEVIATION -INF 0.0151 +INF 


MARGINAL 


-1.0000 
-0.0022 


MARGINAL 


bi. 


The Alternative Model's First Tier. 


GPNLP2 
SOLUTION REPORT SOLVE GPNLP2 USING NLP FROM LINE 193 
SOLVE SUMMARY 
MODEL GPNLP2 OBJECTIVE DEVIATION 
TYPE NLP : DIRECTION MINIMIZE 
SOLVER MINOSS5 FROM LINE 193 
¥X** SOLVER STATUS “ 1 NORMAL COMPLETION 
***% MODEL STATUS 2 LOCALLY OPTIMAL 
*xXx% OBJECTIVE VALUE 0.0 
RESOURCE USAGE, LIMIT 0.419 1000.000 
ITERATION COUNT, LIMIT 4 1000 
EVALUATION ERRORS 0 0 
MINOS --- VERSION 5.0 #£=APR) 1984 


= -_- -_= 
= =- - 


COURTESY OF B. A. MURTAGH AND M. A. SAUNDERS, 


DEPARTMENT OF OPERATIONS RESEARCH, 
STANFORD UNIVERSITY, 
STANFORD CALIFORNIA 94305 U.S.A. 


HORK SPACE NEEDED CES TINAT Em =— 330 WORDS. 
WORK SPACE AVAILABLE aS 454 WIORDS. 
CMAXIMUM OBTAINABLE -- 164974 WORDS.) 


EXIT -- OPTIMAL SOLUTION FOUND 
MAJOR ITERATIONS G6 
HORM RG 7 NORM PI 0.000E+00 
TOTAL USED 
MINOSS TIME 


EQU 
EQU 
EQU 


OBJDEF 
TLIMIT 
DLIMIT 


Al 


VAR 


VAR 


OBJDE 
TLINI 
DET) 


LOWE 
0.7 


DEVIA 


VAR TPOS 


VAR 
VAR 
VAR 


THEG 
BPPOS 
BPNEG 


0.43 UNITS 
0.39 CINTERPRETER - 0.01) 
LOWER LEVEL UPPER MARGINAL 
F ; -~1.0000 
T 30.0000 30.0000 30.0000 ERS 
T -26.7696 —-€26./696 -26.7696 Els 
ACHIEVEMENT FUNCTION 
PEME CITT 
DISTANCE LIMIT 
TIME TO BE SPENT INSPECTING EACH TASK IN BATTERY 
R LEVEL UPPER MARGINAL 
070 1.8000 1.8000 
LOWER LEVEL UPPER MARGINAL 
TION ~INF ; +INF : 
: : +INF 1.0000 
28.2000 +INF 5 
: +INF 600.0000 
: : +INF EPS 


Lig 


c. The Alternative Model's Second Tier. 


GPHLP2 


SOLUTION REPORT SOLVE GPNLP2 USING NLP FROM LINE 191 


SOLVE SUMMARY 
MODEL GPNLP2 OBJECTIVE DEVIATION 
TYPE NLP DIRECTION MINIMIZE 
SOLVER MINOSS FROM LINE 191 

xxxx SOLVER STATUS "1 NORMAL COMPLETION 


**XX MODEL STATUS 


2 LOCALLY OPTIMAL 
*RRM OBJECTIVE VALUE 0.0 


RESOURCE USAGE, LIMIT 0.582 1000.000 
ITERATION COUNT, LIMIT 25 1000 
EVALUATION ERRORS 0 0 


fer nN 0S --- 


VERSION 5.0 APR 1984 


COURTESY OF B. A. MURTAGH AND M. A. SAUNDERS, 
DEPARTMENT OF OPERATIONS RESEARCH, 
STANFORD UNIVERSITY, 


STANFORD CALIFORNIA 94305 U.S.A. 

HORK SPACE NEEDED CESTIMATE) -- 604 WORDS. 

VIORK SPACE AVAILABLE == 838 WORDS. 

(MAXIMUM OBTAINABLE -- 164974 WORDS.) 

EXIT -- OPTIMAL SOLUTION FOUND 

MAJOR ITERATIONS Gl 

NORM RG 7 NORM PI 0.000E+00 

TOTAL USED 0.60 UNITS 

MINOS5 TIME 0.51 CINTERPRETER - 0.06) 

LOWER LEVEL UPPER MARGINAL 

See GU OBJDEF : : : -1.0000 
---- EQU TLIMIT 28.2000 28.2000 28.2000 EPS 
eee ea DLIMIT -186.24997 -186.2497 -186.24597 EPS 

OBJDEF ACHIEVEMENT FUNCTION 

TLIMIT JIMNE LIMIT 

DLIMIT DISTANCE LIMIT 
---- VAR X TIME TO BE SFENT INSPECTING EACH TASK IN BATTERY 

LOWER LEVEL UPPER MARGINAL 
Gl 0.7070 2.8000 2.8000 EPS 
G3 0.7070 1.0000 1.0000 EPS 
AG 2.0200 8.9000 8.9000 ; 
AG 0.7070 3.4000 3.4000 EPS 
$1 0.7070 3.0000 3.0000 EPS 
52 0.7070 2.0000 2.0000 EPS 
$3 0.7070 3.5000 3.5000 EPS 
LOWER . LEVEL UPPER MARGINAL 

---- VAR DEVIATION -INF + IHF : 
Pee VAP TPOS ; ; +I 1.0000 
---- VAR THEG 3.6000 +IHF : 
P=-- VAR BPPOS . + IF 600.0000 
---- VAR BPNEG +INF EPS 


ihe Bey) 


qd. The Alternative Model’s Thitsa Tater 


GPNLP2 
SOLUTION REPORT SOLVE GPNLP2 USING NLP FROM LINE 208 
Se0cle ye SUMMARY 
MODEL GPNLP2 OBJECTIVE DEVIATION 
TYPE NLP . DIRECTION MINIMIZE 
SOLVER WMINOSS5 FROM LINE 208 
RR SOLVER STATUS "1 NORMAL COMPLETION 
RRM MODEL STATUS 2 LOCALLY OPTIMAL 
RRR OBJECTIVE VALUE 4.8104 
RESOURCE USAGE, LIMIT 0.309 1000.000 
ITERATION COUNT, LIMIT eo 1000 
EVALUATION ERRORS 0 0 
MI NO SS” <== | VERSION Mtoe APR 1984 


COURTESY OF B. A. MURTAGH AND M. A. SAUNDERS, 
DEPARTMENT OF OPERATIONS RESEARCH, 
STANFORD UNIVERSITY, 
STANFORD CALIFORNIA 94305 USA. 


MORK SPACE NEEDED CESTIMATE) -- SLESHOR DS. 
WORK SPACE AVAILABLE = 709 WORDS. 
CMAXIMUM OBTAINABLE -- 169974 WORDS.) 
EXIT -- OPTIMAL SOLUTION FOUND 
MAJOR ITERATIONS 13 
HORM RG 7 NORM PI 8.463E-08 
TOTAL USED 0.52 UNITS 
MINOSS TIME 0.25 CINTERPRETER - 0.03) 
LOWER LEVEE UPPER MARGINAL 
- sae OU Op DEF ; -1.0000 
~= 5 Seale al 1 5.6000 3.6000 3.6000 -1.0000 
~-==SEQUe DELIMIT =7 296 -7 ea = Tees ork -1.0000 
OBJDEF ACHIEVEMENT FUNCTION 
TLIMIT TIME LIMIT 
DLIMIT DISTANCE LIMIT 
~~=— VAR XxX TIME TO BE SPENT INSPECTING EACH TASK IN BATTERY 
LOVIER LEVEL UPPER MARGINAL 
G2 O70 710 1.0504 3.2000 EPS 
G4 0.7070 1.4691 1.6000 : 
A2 0.70/70 7 508 4.8000 EPS 
A3 0.7070 1.2784 1.7000 EPS 
34 U0 770 1.2390 1.4000 EPS 
a LOWER LEVEL UPPER MARGINAL 
~--- VAR DEVIATION -INF 4.8104 eee : 
=-== VAR TOs 2.1877 eee Le EPS 
-~--- VAR THEG : : + IF 1.0000 
~~--- VAR BPPOS : 2.6227 + INF : 
~~ == SVARAPENEG + INF 1.0000 


a. 


APPENDIX F 


GLOSSARY 


Human Reliability Terminology 


"Consequence of interest (COI)" is analogous to 
measure of effectiveness (MOE). MOE is used when 
the concern is maximizing effectiveness of some 
positive performance whereas COI is used when the 
concern is minimizing consequences of some 
negative performance (for example: human error). 


"Human error" is any member of a set of human 
actions that exceeds some limit of acceptability. 
The set of actions must be defined and the 
tolerance limits must be established for there to 
be human error. [Ref 10, p.6] 


"Human error probability" is the chance that a 
given task will not be successfully completed by 
personnel within a required minimum time (if a 
time requirement exists). This is the reciprocal 
of human reliability. 


"Human reliability" is the probability that a task 
will be completed successfully by personnel within 
a specified minimum time (if a time requirement 
exists). [Ref 1, p. 221] 


"Worker" is the human on the production line who 
performs the error-prone tasks that potentially 
effect the finished product. A mistake by a 
worker that potentially leads to a flawed product 
is referred to as an "error". 


"Inspector" is the human tasked to find faulty 
finished products before they leave the production 
line. A mistake by an inspector that leads to a 
flawed product is referred to as a "failure". 


"System error" is a worker error which evades 
detection by the inspector and has a cost that is 
of consequence of interest to the system. A 
system error is also referred to as a "flawed 
product". 


B. 


Artillery Terminology 


A "chief" is the senior man in a crew of men who 
operate a gun. 


"Fire(s)", when used as a noun, refers to the 
shooting of artillery munitions. 


A "fire mission" is an order from higher authority 
to shoot one or more rounds of artillery at a 
target. 


A "gun" refers to a Single artillery cannon weapon 
system. 


"Gunnery" refers to the group of skills required 
of crewmen to fire their weapon. 


"Maneuver", when used as a noun, refers to 
infantry or armor elements to which the artillery 
provides fire support. 


A “round" is synonymous with the term projectile. 
A "unit" refers to a battery of artillery. A 


battery is either six or eight cannon weapon 
systems. 


APPENDIX G 
SURVEYS ADMINISTERED TO RESPONDENTS 

The collection of the expert judgments from operational 
personnel was key to solving the illustrative problem. The 
error probability survey and the time use survey were the two 
Surveys given to the respondents. Respondents were briefed 
in person prior to working on the survey. The surveys took 
approximately 45 minutes to complete. 

The error probability survey requires a gun chief to 
compare pairs of errors as to their relative likelihood of 
occurrence. The comparisons are made among Similar type 
errors within the hierarchical areas defined in Figure 9 in 
Appendix A. The gun chief 1S also required to make one direct 
numerical estimate on the likelihood that his section sets any 
type of error. The process by which these survey results are 
turned into probabilities is discussed in Appendix A. The 
method by which the respondent's judgments are checked for 
consistency is discussed in Appendix D. 

The time use survey requires a gun chief to compare the 
amount of time taken to perform one inspection task relative 
another. Comparisons are made among similar type tasks within 
hierarchical areas defined in Figure 9 in Appendix A. One 
inspection task which is routinely done in a relatively 


uniform manner among all gun chiefs is the inspection of the 


2d 


gunner's sight picture on the gun's aiming reference point. 
It is assumed that this task takes 2.0 seconds to perform. 
Using that known value, all other visual inspection tasks are 
scaled accordingly. The process by which the tasks are scaled 


from the survey results is discussed in Appendix C. 


1. The Error Probability Survey. 
FREQUENCY BY WHICH ERRORS OCCUR 
(gunner, asst gunner, ammo handlers, FDC) 

Which crew element (the one on the left or the one on the 
right) is most likely to make an error that you must have 
corrected prior to firing? How many times more likely is the 
more error-prone element to make an error than the less error- 
prone element? (Assume daylight firing, shell HE, fuze TIME, 
Low Angle; with collimator, gunner's quadrant, and safety T.) 

( make one "X" per comparison below ) 


WHICH IS MORE LIKELY ? 


eweweeweeeweeweweweweeweeeweeeeeeewwevwsewewewewewewew wees eB PB estes ewe ewe ew ewe ew ew BF we we we = = 


how many times 
ERROR as likely? ERROR 
lig 7 ey Ae ee) ee 2ueom a5 6 7 8 59 in: 














gunner | | | | == 7 asst 
is SC) S00 es Se 

gunner | | | 1 | | | fe | | | | | | ammo 
|| | handlers 
gunner | | | | | | | ee | | | | | | FDC / 
es el eee | eee || ey ||| safety T 
asst bo SS ee | foe i I ammo 
ae | eee Pee eee ee | ||__ |__| |__| handlers 
est TE IIR ree 
ear eee | ees eee | ee ee | ey | eee ||| [2 ||_||_safety T 
ammo Peete heeft i i 4 | | FDC / 
hangiers| | |_| |_1t_|__I_|_!_]_|_|_|_|__|_safety 7 


Indicate below the number of "good" missions to each 
mission that you must get involved in making a correction on 
either the gunner, asst gunner, ammo handlers, or FDC (a 
"faulty" mission is one in which you must get involved to keep 


shooting accurate. ) 


| How many times as likely 
are good missions to each faulty mission? 

1 | i 2ecee4ee os. 6G 7 8 9 1020730 40850 60 970 
gs bea aU 
Siieiotel | | | | | good 
mission |__|__|__|__|__l__l__]__I__I__]_I_I_|__|__Imission _ 


le ee 


- = -—e@e2eeeeee = 





GUNNER ERRORS | 
Which error (the one on the left or the one on the right) is 
most likely to occur (prior to your safety inspection) in your gun 
section? How many more times likely is the more frequent event 
than the less frequent event? (Assume daylight firing, good 
conditions, shell HE, fuze TIME, Low Angle; with collimator, 
Gunner's quadrant, and safety T.) 


( make one "X" per comparison below ) 
( Mark only six "X"s on this page ) 


WHICH IS 
MORE LIKELY ? 
how many times 
ERROR as likely? ERROR 
in: >9 8 @aG 5 9°4e5c ez 2 3 4 5 6 "7 jee Ie 


Sight == bubbles 
picture 
sight | | Ff Ff fo Eh ee ees 
epl(eie pers: counter 
Si gine es es ee es en en piece 
picture cornmm 
bubbles | | | |! | | (| |==)°0 9] 9 eee 
counter 
bubbles | | | | | | |. (Sse fe) |S) eee 
corpm 
alse a se es ce 
counter corrn 


ASSISTANT GUNNER ERRORS 
Which error (the one on the left or the one on the right) is 
most likely to occur (prior to your safety inspection) in your gun 
section? How many more times likely is the more frequent event 
than the less frequent event? (Assume daylight firing, good 
conditions, shell HE, fuze TIME, Low Angle; with collimator and 
gunner's quadrant. ) 


( make one "X" per comparison below ) 
( Mark only six "X"s on this page ) 


WHICH IS 
MORE LIKELY ? 
how many times 
ERROR as likely? ERROR 
meme oS S& 7 6 5&§ 4 3 2 2 3 4 5 6 7 8 399 ine 


counter i elev 
bubble 
counter | | | | {| | {| J==l-Y I | td | | ex ivi 
bubble 
meumter is | |; | | | ¢ fts=!t | | ded t |) “piece 
COrc”n 
“eo 686d) hCUm;TCUC;TtCtC;TtCTttC]tC( i‘LtCé*dCH==]~COU]TtC<Ci*dSYtéti‘(aisé‘i;;é‘<i‘i;;*‘“‘;«i‘};S*‘Cé‘SNL!StSTS er (1Vv1 
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AMMUNITION HANDLERS ERRORS 
Which error (the one on the left or the one on the right) re 
most likely to occur (prior to your safety inspection) in your gun 
section? How many more times likely is the more frequent event 
than the less frequent event? (Assume daylight firing, good 
conditions, shell HE, fuze TIME, Low Angle; with collimator, 
gunner's quadrant, and safety T.) 


( make one "X" per comparison below ) 
( Mark only six "X"s on this page ) 


WHICH IS 
MORE EERE LY =: 


how many times 


ERROR as likely? ERROR 
in: >9 8 9)7 “6.5 4c eee 2 3 4 5 G6 7) See ian 
charge == shell 
type 
charge | | } | | | | t==] | | | |) eee 
type 
charge | | | | | | | Jee | | 0) 9) eee 
setting 
shell | | | 7 ¢ | | tee h} 7) 5S Seite 
type type 
shell | | [| | | | | tse} (1 | 0) 55 55) Steere 
type setting 
fuze | | | | {J | tee) ee eee 
type | | setting 
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FDC DATA ERRORS 


(data is off of Safety T) 


Which error (the one on the left or the one on the right) 
is most likely to occur? How many more times likely is the more 
frequent event than the less frequent event? (Assume daylight 
firing, good conditions, shell HE, fuze TIME, Low Angle; with 
collimator, gunner's quadrant, and safety T.) 

( make one "X" per comparison below ) 
( Mark only six "X"s on this page ) 


WHICH IS 
MORE LEKELY 7 
how many times 
ERROR as likely? ERROR 
meee 9 6hU8lUGCCFlCGHUUCSUHUC KULLU Bll Fi 2) 01s) 6 ee sa ab abjay 


deflec- == quad- 
Ten) rant 
deflec- | SS | a i es ne eee: ae Oe ~ time for 
tion fuze 
Secs | |; | | | | | ¢eet |F | | | 1 1 ammo 

On chg/sh/fz 
mo | | | | | | | ‘fee; | ¢ | | | | | time for 
rant fuze 
quad- | i a ae a ae ce Poi aio. | | | ~ ammo 

rant chg/sh/fz 
memefor|; | ;} | | | | Je=t | | | t | | | ammo: 
fuze | chg/sh/fz 


2. The Time Use Survey. 
COMPARISON OF TIME TAKEN TO PERFORM 
SAFETY CHECKS ON CREW 
Which check (the one on the left or the one on the right) 
consumes more of your time? How many more seconds does it take to 
make the longer check verses the shorter check in each of the six 
comparisons listed below? (Assume daylight firing, good 
conditions, shell HE, fuze TIME, Low Angle; with collimator, 
gunner's quadrant, and safety T.) 
( make one "X" per comparison below ) 


WHICH CHECK 2S MORE 
TIME CONSUMING 


increased number of seconds it takes to do one task vs the other 


#sec >8 7 6 594 S20 12 83 4 5 6 YF >8 gee 
gunner == asst 
gunner 
gunner | es ee ee es ne es nr TM Te | | ay. 
handlers 
gunner | wm | to YT LS 0 | ee safety T 
asst | | | | | | of Jest 7 | 19 (9) ees 
gunner handlers 
asst | | ¢ | | | 4° 922) 9 ee Sater 7 
gunner 
ammo [= | | | |. | |) 22 a ee eee safety T 
handlers 
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DEFLECTION CHECKS 
Which check (the one on the left or the one on the right) 
consumes more of your time? How many more seconds does it take to 
make the longer check verses the shorter check in each of the six 
comparisons listed below? (Assume daylight firing, good 
conditions, shell HE, fuze TIME, Low Angle; with collimator, 
gunner's quadrant, and safety T.) 
( make one "X" per comparison below ) 
( only mark six "X"s on this form ) 


WHI GH GHECKA ITS MORE 
TIME CONSUMING 


increased number of seconds it takes to do one task vs the other 
Mmeec >8 7 6 5 4 3 2 1 123 4 5 6 7 o>8 #sec 


sight == bubbles 
picture 
a a a ee ne ee ee es Oe ee es 
picture counter 
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picture COrrn 
bubbles | | | | | | | deer | J | | |__| at 
counter 
mmepress|) | | | | | | ~+%Je=i | | | | | CL | piece 
corrn 
ef a i tet tTlhCU|!CUVeey)UCUTUCUSC<‘ES;é‘SHH]Sté‘<“‘*];‘<‘)!S*S*SYY épdece 
counter corrn 
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AMMUNITION CHECKS 


Which check (the one on the left or the one on the right) 
consumes more of your time? How many more seconds does it take to 
make the longer check verses the shorter check in each of the six 
comparisons listed below? (Assume daylight firing, good 
conditions, shell HE, fuze TIME, Low Angle; with collimator, 
gunner's quadrant, and safety T.) 

( make one "X" per comparison below ) 


WHICH CHECK SiSeNORE 
TIME CONSUMING 


increased number of seconds it takes to do one task vs the other 


#SseC >.Beee/amb)08 4 3.825 1 2 3 4 5 647 OS Sioa 
charge = time 
setting 
charge | | | [| | | | sel § |) | | 9) S55 eee 
type 
charge | | | | | | | Je=!l I | | |) |S Sehr 
type/wt/lot 
time | | | | | oL | tS) 9) | 
setting type 
time | | | [| | { | tee) (1 | SS eee 
setting type/wt/lot 
fuze | | + | | {| | Jes] 4) {9h Se) 5 Sere 
cype type/wt/lot 


ie fen) ee PO) oe | | AC ee 
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FDC DATA CHECKS 
(Safety T violations) 


Which check (the one on the left or the one on the right) 
consumes more of your time? How many more seconds does it take to 
make the longer check verses the shorter check in each of the six 
comparisons listed below? (Assume daylight firing, good 
conditions, shell HE, fuze TIME, Low Angle; with collimator, 
gunner's quadrant, and safety T.) 

( make one "X" per comparison below ) 


WHICH CHECK IS MORE 
TIME CONSUMING 


increased number of seconds it takes to do one task vs the other 


Meece>o 7 6 5 43 2 1 123 4 5 6 7 o8 #sec 
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quad- me | Cll i le | Th t==].C Se eCSC*SdTC(‘<itsés‘SEC*dLSCCtimee:s« tor 
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qQuad- iss iomi | joi (eet) | | | ¢ ¢ 7] ammo: 
rant chg/sh/fz 
mamenter,; | | | «| | | |J==| | | | | Y | J ammo: 
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